Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

What is the output output process of MapReduce

Shulou Source: shulou.com Published: 2022-06-01 06:00:09 11月14日 Update

This article mainly explains "what is the output output process of MapReduce". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "what is the output output process of MapReduce"?

1. First, take a look at the execution entry / /-- ReduceTask.javapublic void run (JobConf job, TaskUmbilicalProtocol umbilical) throws IOException, InterruptedException, ClassNotFoundException {job.setBoolean ("mapreduce.job.skiprecords", this.isSkipping ()); if (this.isMapOrReduce ()) {this.copyPhase = this.getProgress (). AddPhase ("copy") This.sortPhase = this.getProgress (). AddPhase ("sort"); this.reducePhase = this.getProgress (). AddPhase ("reduce");} TaskReporter reporter = this.startReporter (umbilical); boolean useNewApi = job.getUseNewReducer (); / / reducetask initialization work this.initialize (job, this.getJobID (), reporter, useNewApi); if (this.jobCleanup) {this.runJobCleanupTask (umbilical, reporter) } else if (this.jobSetup) {this.runJobSetupTask (umbilical, reporter);} else if (this.taskCleanup) {this.runTaskCleanupTask (umbilical, reporter);} else {this.codec = this.initCodec (); RawKeyValueIterator rIter = null; ShuffleConsumerPlugin shuffleConsumerPlugin = null; Class combinerClass = this.conf.getCombinerClass (); CombineOutputCollector combineCollector = null! = combinerClass? New CombineOutputCollector (this.reduceCombineOutputCounter, reporter, this.conf): null; Class

Tags: Object method output that is run process result up and down context that is content delimiter path logic creative reflective learning working practical deeper Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno MySQL Docker Shulou Tech Info Shulou Information OPPO Reno