How to compile spark1.3 on Linux64 bit operating system (CentOS6.6)
1. Execute the following command after downloading 1.3.0 source code on the official website:
. / make-distribution.sh-tgz-skip-java-test-with-tachyon-Dhadoop.version=2.4.0-Djava.version=1.7-Dprotobuf.version=2.5.0-Pyarn-Phive- Phive-thriftserver
two。 Parameter description:
-- tgz generates deployment packages
-- skip-java-test Filtration Test Phase
With-tachyon feels that tachyon is the trend, so add the support of tachyon
-Dhadoop.version specifies the version of hadoop. The default is 1.0.4.
-Djava.version specifies the java version. Default is 1.6.
-Dprotobuf.version specifies the version of protobuf. The default is 2.4.1. It is specified here because of the 2.5.0 used by hadoop. If you do not modify it, there will be problems with the matching.
-Pyarn-Phive-Pthriftserver is supported by yarn and hive respectively.
3. BUILD SUCCESS appears more than an hour later, that is, the compilation is successful, and the compiled files can be found in the spark-1.3.0 folder.