Hadoop installation 1.0 (short version)
Premise:
Make sure iptables is off and selinux is disabled
1. Prepare hardware
1 namenode and 3 datanodes
namenode 192.168.137.100
datanode1 192.168.137.101
datanode2 192.168.137.102
datanode3 192.168.137.103
2. Create hadoop users on 4 machines (or other user names)
useradd hadoop
Install JDK 1.6 on 4 machines
Java_HOME is installed in/jdk
Configure environment variables
vim /etc/bashrc
export JAVA_HOME=/jdk
scp -r /jdk* datanode1:/
scp -r /jdk* datanode2:/
scp -r /jdk* datanode3:/
4. Configure 4 machines for multi-machine mutual trust
Make sure you remember to compare the number of nodes
/home/hadoop/.ssh
and all files below are set to 700 permissions
5. Install hadoop
tar zxvf hadoop-1.0.4.tar
Installed in/hadoop
Set the/hadoop permission position to 755
vim /hadoop/conf/hadoop-env.sh
export JAVA_HOME=/jdk
vim /hadoop/conf/core-site.xml
hadoop.tmp.dir
/home/hadoop/tmp
fs.default.name
hdfs://namenode:9000
vim /hadoop/conf/mapred-site.xml
mapred.job.tracker
namenode:9001
vim /hadoop/conf/hdfs-site.xml
dfs.name.dir
/home/hadoop/name
dfs.data.dir
/home/hadoop/name
dfs.replication
2
vim /hadoop/conf/masters
192.168.137.100
vim /hadoop/conf/slaves
192.168.137.101
192.168.137.102
192.168.137.103
Copy the configured HADOOP to the datanode
cd /
scp -r hadoop datanode1:/hadoop
scp -r hadoop datanode2:/hadoop
scp -r hadoop datanode3:/hadoop
6. Install zookeeper
tar zxvf zookeeper-3.3.4.tar
Installed in/zookeeper
cd /zookeeper/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
join
dataDir=/zookeeper-data
dataLogDir=/zookeeper-log
server.1=namenode:2888:3888
server.2=datanode1:2888:3888
server.3=datanode2:2888:3888
server.4=datanode3:2888:3888
Create/zookeeper-data
mkdir /zookeeper-data
Create/zookeeper log
Create file/zookeeper-data/myid
vim /zookeeper-data/myid
1
(Write 2 in datanode1)
(Write 3 in datanode2)
(Write 4 in datanode3)
10. Install hive
tarzxvf hive-0.8.0.tar
to/hive
vim /hive/bin/hive-config.sh
export HADOOP_HOME=/hadoop
export PATH=.$ HADOOP_HOME/bin:$PATH
export HIVE_HOME=/hive
export PATH=$HIVE_HOME/bin:$PATH
export JAVA_HOME=/jdk
export JRE_HOME=/jdk/jre
export CLASSPATH=.$ JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=.$ JAVA_HOME/bin:$JRE_HOME/bin:$PATH
vim /etc/bashrc
export HIVE_HOME=/hive
11. Start hadoop
Format and boot the system
su hadoop
cd /usr/local/hadoop/bin
./ hadoop namenode -format
./ start-dfs.sh
./ start-mapred.sh
http://192.168.137.100:50070 View HDFS namenode
http://192.168.137.100:50030 View MAPREDUCE JOB TRACKERS
http://192.168.137.101:5006 View TASK TRACKER on datanode1
12. Related Orders
hadoop fs -mkdir direc
hadoop fs -ls
hadoop fs -cp file:///tmp/test.file /user/hadoop/direc
