Shulou(Shulou.com)06/03 Report--

The installation of Scala Scala before installing spark has been introduced in the hadoop installation configuration

1. Download the spark installation package at the following address

Http://spark.apache.org/downloads.html

What I chose is

Spark-1.4.1-bin-hadoop2.6.tgz on / root/software

Decompression

Tar zxvf spark-1.4.1-bin-hadoop2.6.tgz

2. Configure system environment variables

Vim / etc/profile

Export SPARK_HOME=/root/sherry/spark-1.4.1

Export PATH=$SPARK_HOME/bin:$PATH

Source / etc/profile

3. Configure the spark file

1) [root@HBMS-DKXD-DEV sherry] # cd spark-1.4.1-bin-hadoop2.6/

[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] # cd conf

[root@HBMS-DKXD-DEV conf] # vim spark-env.sh

Add the following

Export SCALA_HOME=/root/sherry/scala-2.10.4

Export JAVA_HOME=/usr/java/jdk1.7.0_45

Export SPARK_WORKER_MEMORY=1g

Export SPARK_MASTER_IP=10.118.46.22

2) configure slave

I don't have my own configuration.

4. Start the cluster

[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] #. / sbin/start-all.sh

Close

[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] #. / sbin/stop-all.sh

Check whether the installation is successful

Jps

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.