Example Analysis of null pointer exception in hive join
Editor to share with you an example of hive join null pointer exception analysis, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!
2015-07-28 10 0%Ended Job 0315 Stage-2 map = 100%, reduce = 0%Ended Job = job_1437720498561_1035 with errorsError during job Obtaining debugging information...Examining task ID: task_1437720498561_1035_m_000000 (and more) from job job_1437720498561_1035Task with the most failures (4):-Task ID: task_1437720498561_1035_m_000000URL: http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1437720498561_1035&tipid=task_1437720498561_1035_m_000000-----Diagnostic Messages for this Task:Error: java.lang.RuntimeException: java.lang.NullPointerException at Org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map (ExecMapper.java:172) at org.apache.hadoop.mapred.MapRunner.run (MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper (MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run (MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run (YarnChild.java:163) At java.security.AccessController.doPrivileged (Native Method) at javax.security.auth.Subject.doAs (Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs (UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main (YarnChild.java:158) Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable (MapJoinOperator.java:286) at org. Apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp (MapJoinOperator.java:313) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged (Operator.java:1069) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged (Operator.java:1073) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged (Operator.java:1073) at org.apache.hadoop.hive.ql.exec.MapOperator. Process (MapOperator.java:501) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map (ExecMapper.java:163)... 8 moreFAILED: Execution Error Return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
If NullPointerException occurs during hive join operation, please execute set hive.auto.convert.join=false
Setting the value of hive.auto.convert.join to true,hive will automatically use MapJoin when the size of the table matches the setting.
The above is all the contents of the article "sample Analysis of hive join null pointer exception". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
