Spark installation configuration
The installation of Scala Scala before installing spark has been introduced in the hadoop installation configuration
1. Download the spark installation package at the following address
Http://spark.apache.org/downloads.html
What I chose is
Spark-1.4.1-bin-hadoop2.6.tgz on / root/software
Decompression
Tar zxvf spark-1.4.1-bin-hadoop2.6.tgz
2. Configure system environment variables
Vim / etc/profile
Export SPARK_HOME=/root/sherry/spark-1.4.1
Export PATH=$SPARK_HOME/bin:$PATH
Source / etc/profile
3. Configure the spark file
1) [root@HBMS-DKXD-DEV sherry] # cd spark-1.4.1-bin-hadoop2.6/
[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] # cd conf
[root@HBMS-DKXD-DEV conf] # vim spark-env.sh
Add the following
Export SCALA_HOME=/root/sherry/scala-2.10.4
Export JAVA_HOME=/usr/java/jdk1.7.0_45
Export SPARK_WORKER_MEMORY=1g
Export SPARK_MASTER_IP=10.118.46.22
2) configure slave
I don't have my own configuration.
4. Start the cluster
[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] #. / sbin/start-all.sh
Close
[root@HBMS-DKXD-DEV spark-1.4.1-bin-hadoop2.6] #. / sbin/stop-all.sh
Check whether the installation is successful
Jps