Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

How to compile spark1.3 on Linux64 bit operating system (CentOS6.6)

Shulou Source: shulou.com Published: 2022-06-03 08:04:38 11月23日 Update

1. Execute the following command after downloading 1.3.0 source code on the official website:

. / make-distribution.sh-tgz-skip-java-test-with-tachyon-Dhadoop.version=2.4.0-Djava.version=1.7-Dprotobuf.version=2.5.0-Pyarn-Phive- Phive-thriftserver

two。 Parameter description:

-- tgz generates deployment packages

-- skip-java-test Filtration Test Phase

With-tachyon feels that tachyon is the trend, so add the support of tachyon

-Dhadoop.version specifies the version of hadoop. The default is 1.0.4.

-Djava.version specifies the java version. Default is 1.6.

-Dprotobuf.version specifies the version of protobuf. The default is 2.4.1. It is specified here because of the 2.5.0 used by hadoop. If you do not modify it, there will be problems with the matching.

-Pyarn-Phive-Pthriftserver is supported by yarn and hive respectively.

3. BUILD SUCCESS appears more than an hour later, that is, the compilation is successful, and the compiled files can be found in the spark-1.3.0 folder.

Tags: Version compilation file support success parameter available in command hour sense folder time source code trend problem phase test generation operating system system Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno NVidia Xiaomi Apple macOS MySQL