How to install shark
This article mainly introduces how to install shark, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.
Spark0.9.1 installation
Decompress
Modify directory user rights, user hadoop login
Modify the configuration file
/ / slaves
Hadoop1
Hadoop2
Hadoop3
/ / spark-env.sh
Export SPARK_MASTER_IP=hadoop1
Export SPARK_MASTER_PORT=7077
Export SPARK_WORKER_CORES=1
Export SPARK_WORKER_INSTANCES=1
Export SPARK_WORKER_MEMORY=3g
Scp to a node
* * *
* * *
Shark installation
Download source code and decompress
Source code compilation
SHARK_HADOOP_VERSION=2.2.0 SHARK_YARN=true sbt/sbt assembly
Modify the configuration file
/ / shark-env.sh
Export SPARK_MEM=2g
# (Required) Set the master program's memory
Export SHARK_MASTER_MEM=1g
# (Optional) Specify the location of Hive's configuration directory. By default
# Shark run scripts will point it to $SHARK_HOME/conf
Export HIVE_CONF_DIR= "/ app/hadoop/hive013/conf"
# For running Shark in distributed mode, set the following:
Export HADOOP_HOME= "/ app/hadoop/hadoop220"
Export SPARK_HOME= "/ app/hadoop/spark091"
Export MASTER= "spark://hadoop1:7077"
Scp to a node
* * *
* * *
Start hive metasotre service
Nohup bin/hive-- service metastore > metastore.log 2 > & 1 &
Start shark
Bin/shark
. / bin/shark-service sharkserver
. / bin/shark-h-p
Thank you for reading this article carefully. I hope the article "how to install shark" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!
