Ultra-detailed explanation of Hadoop2 configuration
1. Cluster environment
Master 192.168.2.100
Slave1 192.168.2.101
Slave2 192.168.2.102
two。 Download the installation package
# Master
Wget http://mirrors.shu.edu.cn/apache/hadoop/common/hadoop-2.8.4/hadoop-2.8.4.tar.gz
Tar zxvf hadoop-2.8.4.tar.gz
3. Modify Hadoop configuration file
# Master
Cd hadoop-2.8.4/etc/hadoop
Vim hadoop-env.sh
Export JAVA_HOME=/usr/local/src/jdk1.8.0_152
Vim yarn-env.sh
Export JAVA_HOME=/usr/local/src/jdk1.8.0_152
Vim slaves
Slave1
Slave2
Vim core-site.xml
Fs.defaultFS hdfs://master:9000 hadoop.tmp.dir file:/usr/local/hadoop-2.8.4/tmp
Vim hdfs-site.xml
Dfs.namenode.secondary.http-address master:9001 dfs.namenode.name.dir file:/usr/local/hadoop-2.8.4/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop-2.8.4/dfs/data Dfs.repliction 3
Vim mapred-site.xml
Mapreduce.framework.name yarn
Vim yarn-site.xml
Yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapreduce.shuffle.class org.apache.hadoop.mapred.ShuffleHandler yarn.resourcemanager.address master:8032 yarn.resourcemanager.scheduler.address Master:8030 yarn.resourcemanager.resource-tracker.address master:8035 yarn.resourcemanager.admin.address master:8033 yarn.resourcemanager.webapp.address master:8088
# create temporary directories and file directories
Mkdir / usr/local/hadoop-2.8.4/tmp
Mkdir-p / usr/local/hadoop-2.8.4/dfs/name
Mkdir-p / usr/local/hadoop-2.8.4/dfs/data
4. Configure environment variables
# Master 、 Slave1 、 Slave2
Vim / .bashrc
HADOOP_HOME=/usr/local/src/hadoop-2.8.2
Export PATH=$PATH:$HADOOP_HOME/bin
# refresh environment variables
Source / .bashrc
5. Copy installation package
# Master
Scp-r / usr/local/src/hadoop-2.8.4 root@slave1:/usr/local/src/hadoop-2.8.4
Scp-r / usr/local/src/hadoop-2.8.4 root@slave2:/usr/local/src/hadoop-2.8.4
6. Start the cluster
# Master
# initialize Namenode
Hadoop namenode-format
# start the cluster
. / sbin/start-all.sh
6. Cluster status
Jps
# Master
# Slave1
# Slave2
7. Monitoring web pages
Http://master:8088
test
Shut down the cluster
. / sbin/hadoop stop-all.sh