Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to implement the counter in hadoop. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

1. Define counters

1) enumerate declaration counters

Context context...// custom enumeration variable EnumCounter counter = context.getCounter (Enum enum)

2) Custom counter

Context context...// names groupName and counterNameCounter counter = context.getCounter (String groupName,String counterName) 2, assigns values to counters

1) initialize the counter

Counter.setValue (long value); / / set the initial value

2) counter self-increment

Counter.increment (long incr); / / increase the count 3 to get the value of the counter

1) get the value of the enumeration counter

Job job...job.waitForCompletion (true); Counters counters=job.getCounters (); Counter counter=counters.findCounter ("BAD_RECORDS"); / / find enumerated counters, if the variable of Enum is BAD_RECORDSlong value=counter.getValue (); / / get the count value

2) get the value of the custom counter

Job job...job.waitForCompletion (true); Counters counters=job.getCounters (); Counter counter=counters.findCounter ("ErrorCounter", "toolong"); / / if groupName is ErrorCounter,counterName, toolonglong value=counter.getValue (); / / get the count value

3) get the value of the built-in counter

Textpop-upJob job...job.waitForCompletion (true); Counters counters=job.getCounters (); / / find the counter of the number of reduce started when the job runs. GroupName and counterName can query Counter counter=counters.findCounter ("org.apache.hadoop.mapreduce.JobCounter", "TOTAL_LAUNCHED_REDUCES") from the built-in counter table (listed above); / / if groupName is org.apache.hadoop.mapreduce.JobCounter,counterName, TOTAL_LAUNCHED_REDUCESlong value=counter.getValue (); / / get the count value

4) get the values of all counters

Counters counters = job.getCounters (); for (CounterGroup group: counters) {for (Counter counter: group) {System.out.println (counter.getDisplayName () + ":" + counter.getName () + ":" + counter.getValue ()) }} this is the end of the article on "how to implement counters in hadoop". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.