How to compile Hadoop2.2.0

This article mainly introduces how to compile Hadoop2.2.0, the article introduces in great detail, has a certain reference value, interested friends must read it!

Apache provides the Hadoop2.2.0 32-bit version as well as the source code, but at present, most systems use 64-bit systems, so we need to recompile the 64-bit version using the source code ourselves.

I use the Ubuntu12 version of the system

Commands for installing dependency packages

Sudo port install libtool

Sudo port install cmake

Sudo port install ncurses

Sudo port install openssl

Sudo port install maven3

If you also need to install jdk,jdk, please use version 1.6 or 1.7 (recommended).

Note: there will be problems with the compilation of jdk 1.8. The specific reason is unknown.

You also need to install ssh

Sudo apt-get install openssh-client

Sudo apt-get install openssh-server

Mavensudo apt-get install maven

If you need to generate doc documents at compile time, you need to install findbugs

Http://sourceforge.jp/projects/sfnet_findbugs/releases/

Decompress configuration findbugs

Rcp is used in communication, so protoc is needed.

Https://code.google.com/p/protobuf/downloads/list

You need to modify a configuration in the hadoop source code before compilation (this should be a BUG in hadoop)

Hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/pom.xml

Under line 55, add the following:

fifty-six

Org.mortbay.jetty

Jetty-util

Test

Org.mortbay.jetty

Jetty

Test

Compile command

Generate docs

Mvn package-DskipTests-Pdist,native,docs

Do not generate docs

Mvn package-DskipTests-Pdist,native

Some configuration information in the / etc/profile file when I compiled the source code

Export JAVA_HOME=/usr/app/jdk1.7.0_25

Export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

Export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

Export PATH=/home/ricardo/apache-maven-3.0.5/bin:$PATH

Export FINDBUGS_HOME=/home/ricardo/findbugs-3.0.0

Export PATH=FINDBUGS_HOME/bin:$PATH

Export HADOOP_PROTOC_PAT=/usr/local/bin

Export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

Export FORREST_HOME=/home/ricardo/apache-forrest-0.9

Export PATH=$PATH:$FORREST_HOME/bin

The whole compilation process takes more than 40 minutes, depending on network and computer performance.

The above is all the content of this article "how to compile Hadoop2.2.0". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!