How CentOS7 compiles Hadoop-2.7.2
This article mainly introduces how CentOS7 compiles Hadoop-2.7.2. It is very detailed and has certain reference value. Friends who are interested must finish reading it.
First of all, the 32-bit Linux operating system can download the compiled Hadoop directly, but the 64-bit Linux operating system needs to compile itself.
Later, the official website has provided 64-bit since the hadoop2.5 version. It can be downloaded and used directly.
However, if you do it yourself and have plenty of food and clothing, you can regard it as learning. Here's how to start:
1. Tool preparation, the most reliable are the tools required in the hadoop documentation.
Go to the hadoop website and click source to download hadoop-2.7.2-src.tar.gz.
Decompress it
Tar-zxvf hadoop-2.7.2-src.tar.gz
Get the hadoop-2.7.2-src folder.
Go to the hadoop-2.7.2-src folder and view BUILDING.txt
Cd hadoop-2.7.1-src
Vim BUILDING.txt
You can see the libraries or tools required for compilation:
These include:
JDK1.7+
Maven 3.0 or later
Findbugs 1.3.9
ProtocolBuffer 2.5.0
Cmake 2.6
Zlib-devel
Openssl-devel
In addition to the above, in order not to make mistakes, you actually need autoconf automake gcc and so on.
Let's start preparing these tools:
First use the su command to get root permissions to avoid various restrictions
two。 Download jdk-8u65-linux-x64.tar.gz, extract it and move it to the / opt directory
Tar-zxvf jdk-8u65-linux-x64.tar.gz
Mv jdk1.8.0_65 / opt
Then open / etc/profile to configure the jdk environment variable
Vim / etc/profile
Press I to enter insert mode and add at the end of the file
Export JAVA_HOME=/opt/jdk1.8.0_65
Export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/tools.jar
Export PATH=$PATH:$JAVA_HOME/bin
Export JRE_HOME=/opt/jdk1.8.0_65/jre
Export PATH=$PATH:$JRE_HOME/bin
Press Esc, Shift+:, wq, and press enter to save and exit editing.
Enter source / etc/profile and enter to save the changes.
Run javac-version to check the status:
[root@server1 java] # javac-version
Javac 1.8.0_65
[root@server1 java] #
3. Install various libraries below
Yum-y install svn ncurses-devel gcc*
Yum-y install lzo-devel zlib-devel autoconf automake libtool cmake openssl-devel
4. Install protobuf-2.5.0.tar.gz (note that version must be 2.5.0)
Tar zxvf protobuf-2.5.0.tar.gz
Enter protobuf-2.5.0 and execute in turn.
Cd protobuf-2.5.0
. / configure
Make
Make install
Verify that the installation is complete
Protoc-version
Appear
[root@server1 protobuf-2.5.0] # protoc-- version
Libprotoc 2.5.0
[root@server1 protobuf-2.5.0] #
5. Install maven
Download apache-maven-3.3.9-bin.tar.gz
Extract and configure environment variables
Decompress:
Tar-zxvf apache-maven-3.3.9-bin.tar.gz
Move to the / opt directory:
Mv apache-maven-3.3.9 / opt
Configure environment variables:
Vim / etc/profile
Add at the end:
Export MAVEN_HOME=/opt/apache-maven-3.3.9
Export MAVEN_OPTS= "- Xms256m-Xmx512m"
Export PATH=$PATH:$MAVEN_HOME/bin
Press Esc, Shift+:, wq, and enter to save and edit.
Enter source / etc/profile and enter to save the changes.
Check the installation status: mvn-version
[root@server1 ~] # mvn-version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: / usr/local/maven-3.3.9
Java version: 1.8.0_65, vendor: Oracle Corporation
Java home: / usr/java/jdk1.8.0_65/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-229.el7.x86_64", arch: "amd64", family: "unix"
[root@server1 ~] #
6. Install ant
Download apache-ant-1.9.7-bin.tar.gz
Extract and configure environment variables
Decompress:
Tar-zxvf apache-ant-1.9.7-bin.tar.gz
Move to / opt directory
Mv apache-ant-1.9.7 / opt
Configure environment variables
As above, no addition is made at the end of the / etc/profile file:
Export ANT_HOME=/opt/apache-ant-1.9.7
Export PATH=$PATH:$ANT_HOME/bin
Then save, exit, and make the changes take effect.
View the installation results:
Ant-version
[root@server1 ~] # ant-version
Apache Ant (TM) version 1.9.7 compiled on April 9 2016
[root@server1 ~] #
7. Install findbugs
Download findbugs-3.0.1.tar.gz and select the standard version above
Extract and configure environment variables
Decompress:
Tar-zxvf findbugs-3.0.1.tar.gz
Move to / opt directory
Mv findbugs-3.0.1 / opt
Configure environment variables:
Add at the end of the / etc/profile file:
Export FINDBUGS_HOME=/opt/findbugs-3.0.1
Export PATH=$PATH:$FINDBUGS_HOME/bin
Save the exit and make the changes take effect.
View installation results
[root@server1 ~] # findbugs-version
3.0.1
[root@server1 ~] #
8. Ready to finish, let's start compiling hadoop
Go to the hadoop-2.7.2-src directory
Use the command:
Mvn clean package-Pdist,native-DskipTests-Dtar
Or:
Mvn package-Pdist,native-DskipTests-Dtar
To compile.
Be sure to keep the network open. After a long wait, the following picture shows that the compilation is successful!
Due to problems in the middle of my network, I recompiled it for almost 3 hours.
The compiled file is under / hadoop-dist/target/hadoop-2.7.2.tar.gz.
Note:
1. For all commands, hand tapping is recommended, copy and paste is not recommended, in order to avoid unnecessary errors caused by the space of the page.
two。 Be sure to keep the network open, and if a file is missing, clean up the maven (use the command mvn clean) before recompiling.
3. If the compilation fails due to the same error all the time, it may be that some library or tool is missing. Check that all of the above tools are installed successfully and that the version is correct.
The above is all the content of the article "how CentOS7 compiles Hadoop-2.7.2". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!
