How to realize pseudo-distributed Construction in Hadoop2.4.1
Editor to share with you how to achieve pseudo-distributed Hadoop2.4.1 building, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to understand it!
1. Prepare the Linux environment
Click the VMware shortcut, right open the location of the file-> double-click vmnetcfg.exe-> VMnet1 host-only-> modify the subnet ip setting IP address range: 192.168.1.0 Subnet Mask: 255.255.255.0-> apply-> ok
Go back to windows-> Open Network and sharing Center-> change Adapter Settings-> right-click VMnet1-> Properties-> double-click IPv4-> set the IP:192.168.1.100 subnet mask for windows: 255.255.255.0-> Click OK
On virtual software-- My Computer-> Select virtual machine-> right-click-> settings-> network adapter-> host only-> ok
1.1 modify hostname
Vim / etc/sysconfig/network
NETWORKING=yes
HOSTNAME=itcast #
1.2 modify IP
There are two ways:
First: modify it through the Linux graphical interface (highly recommended)
Enter the Linux graphical interface-> right click on the two minicomputers on the top right-> Click Edit connections-> Select the current network System eth0-> click the edit button-> Select IPv4-> method Select manual-> click the add button-> add the IP:192.168.1.101 subnet mask: 255.255.255.0 Gateway: 192.168.1.1-> apply
Second: modify the configuration file (for losers and programmers only)
Vim / etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE= "eth0"
BOOTPROTO= "static" #
HWADDR= "00:0C:29:3C:BF:E7"
IPV6INIT= "yes"
NM_CONTROLLED= "yes"
ONBOOT= "yes"
TYPE= "Ethernet"
UUID= "ce22eeca-ecde-4536-8cc2-ef0dc36d4a8c"
IPADDR= "192.168.1.101" #
NETMASK= "255.255.255.0" #
GATEWAY= "192.168.1.1" # #
1.3 modify the mapping relationship between hostname and IP
Vim / etc/hosts
192.168.1.101 itcast
1.4 turn off the firewall
# View firewall status
Service iptables status
# turn off the firewall
Service iptables stop
# check the boot status of the firewall
Chkconfig iptables-list
# turn off the firewall and boot
Chkconfig iptables off
1.5 restart Linux
Reboot
two。 Install JDK
2.1 upload
2.2 decompress jdk
# create a folder
Mkdir / usr/java
# decompression
Tar-zxvf jdk-7u55-linux-i586.tar.gz-C / usr/java/
2.3 add java to the environment variable
Vim / etc/profile
# add at the end of the file
Export JAVA_HOME=/usr/java/jdk1.7.0_5
Export PATH=$PATH:$JAVA_HOME/bin
# refresh configuration
Source / etc/profil
3. Install hadoop2.4.1 (itcast01 is the hostname)
Note: hadoop2.x configuration file $HADOOP_HOME/etc/hadoop
Pseudo-distribution requires five configuration files to be modified
3.1Configuring hadoop
The first one: hadoop-env.sh
Vim hadoop-env.sh
# Line 27
Export JAVA_HOME=/usr/java/jdk1.7.0_65
The second one: core-site.xml
Fs.defaultFS
Hdfs://itcast01:9000
Hadoop.tmp.dir
/ itcast/hadoop-2.4.1/tmp
The third one: hdfs-site.xml
Dfs.replication
one
Fourth: mapred-site.xml (mv mapred-site.xml.template mapred-site.xml)
Mv mapred-site.xml.template mapred-site.xml
Vim mapred-site.xml
Mapreduce.framework.name
Yarn
The fifth: yarn-site.xml
Yarn.resourcemanager.hostname
Itcast01
Yarn.nodemanager.aux-services
Mapreduce_shuffle
3.2 add hadoop to the environment variable
Vim / etc/proflie
Export JAVA_HOME=/usr/java/jdk1.7.0_65
Export HADOOP_HOME=/itcast/hadoop-2.4.1
Export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
Source / etc/profile
3.3.Formenting namenode (initializing namenode)
Hdfs namenode-format (hadoop namenode-format)
3.4 start hadoop
Start HDFS first
Sbin/start-dfs.sh
Restart YARN
Sbin/start-yarn.sh
3.5 verify that the startup is successful
Use the jps command to verify
27408 NameNode
28218 Jps
27643 SecondaryNameNode
28066 NodeManager
27803 ResourceManager
27512 DataNode
Http://192.168.8.118:50070 (HDFS Management Interface)
Http://192.168.8.118:8088 (MR Management Interface)
4. Configure ssh login-free
# generate ssh login-free key
# go to my home directory
Cd / .ssh
Ssh-keygen-t rsa (four carriage returns)
After executing this command, two files, id_rsa (private key) and id_rsa.pub (public key), are generated.
Copy the public key to the machine that will not be logged in
Ssh-copy-id localhost
These are all the contents of the article "how to achieve pseudo-distributed Construction in Hadoop2.4.1". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
