How to solve the error report in the operation of SimpleKMeansClustering
This article mainly introduces the relevant knowledge of "how to solve errors in SimpleKMeansClustering operation". The editor shows you the operation process through actual cases. The operation method is simple, fast and practical. I hope this article "how to solve errors in SimpleKMeansClustering operation" can help you solve the problem.
Environment list
The software is explicitly called version.
Hadoop
0.20.2
Mahout
0.4
Eclipse
Kepler Service Release 1
Error code:
ClassNotFoundException: org.apache.mahout.math.function.IntDoubleProcedure
Solution:
At first, I subjectively thought that IntDoubleProcedure was in the mahout-math-0.4.jar package, but it was not in the package after testing.
Later found that IntDoubleProcedure in mahout-collections-1.0.jar, add mahout-collections-1.0.jar this package, will not report the above error.
Contents of the file:
Package com.mahout.cluster;import java.io.File;import java.io.IOException;import java.util.ArrayList;import java.util.List;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.SequenceFile;import org.apache.hadoop.io.Text;import org.apache.mahout.clustering.WeightedVectorWritable Import org.apache.mahout.clustering.kmeans.Cluster;import org.apache.mahout.clustering.kmeans.KMeansDriver;import org.apache.mahout.common.distance.EuclideanDistanceMeasure;import org.apache.mahout.math.RandomAccessSparseVector;import org.apache.mahout.math.Vector;import org.apache.mahout.math.VectorWritable Public class SimpleKMeansClustering {public static final double [] [] points = {{1,1}, {2,1}, {1,2}, {2,2}, {3,3}, {8,8}, {9,8}, {8,9}, {9,9}} Public static void writePointsToFile (List points, String fileName, FileSystem fs, Configuration conf) throws IOException {Path path = new Path (fileName); SequenceFile.Writer writer = new SequenceFile.Writer (fs, conf, path, LongWritable.class, VectorWritable.class); long recNum = 0 VectorWritable vec = new VectorWritable (); for (Vector point: points) {vec.set (point); writer.append (new LongWritable (recNum++), vec);} writer.close ();} public static List getPoints (double [] [] raw) {List points = new ArrayList (); for (int I = 0; I < raw.length; ionization +) {double [] fr = raw [I]; Vector vec = new RandomAccessSparseVector (fr.length) Vec.assign (fr); points.add (vec);} return points;} public static void main (String args []) throws Exception {int k = 3; List vectors = getPoints (points); File testData = new File ("testdata"); if (! testData.exists ()) {testData.mkdir ();} testData = new File ("testdata/points") If (! testData.exists ()) {testData.mkdir ();} Configuration conf = new Configuration (); FileSystem fs = FileSystem.get (conf); writePointsToFile (vectors, "testdata/points/file1", fs, conf); Path path = new Path ("testdata/clusters/part-00000"); SequenceFile.Writer writer = new SequenceFile.Writer (fs, conf, path, Text.class, Cluster.class); for (int I = 0 I < k; iTunes +) {Vector vec = vectors.get (I); Cluster cluster = new Cluster (vec, I, new EuclideanDistanceMeasure ()); writer.append (new Text (cluster.getIdentifier ()), cluster);} writer.close (); KMeansDriver.run (conf, new Path ("testdata/points"), new Path ("testdata/clusters"), new Path ("output"), new EuclideanDistanceMeasure (), 0.001, 10, true, false) SequenceFile.Reader reader = new SequenceFile.Reader (fs, new Path ("output/" + Cluster.CLUSTERED_POINTS_DIR + "/ part-m-00000"), conf); IntWritable key = new IntWritable (); WeightedVectorWritable value = new WeightedVectorWritable () While (reader.next (key, value)) {System.out.println (value.toString () + "belongs to cluster" + key.toString ());} reader.close ();}} about "how to solve errors in SimpleKMeansClustering operation", thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.
