Shulou(Shulou.com)06/01 Report--

This article focuses on "hadoop cdh2.6.0-5.7.0 pseudo-distributed version of how to build", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to build the pseudo-distributed version of hadoop cdh2.6.0-5.7.0.

1 Environmental preparation

1 it is convenient to install rz sz to upload and download files on your local windows

Yum install-y lrzsz

2 the software tools are well equipped

-rw-r--r--. 1 root root 311585484 Feb 20 2018 hadoop-2.6.0-cdh6.7.0.tar.gz

-rw-r--r--. 1 root root 173271626 Feb 23 17:25 jdk-8u45-linux-x64.gz

3 to install the environment

Mkdir-p / usr/local/java

Tar-zxvf jdk-8u45-linux-x64.gz-C / usr/local/java

Mkdir / opt/local

Tar-zxvf hadoop-2.6.0-cdh6.7.0.tar.gz-C / opt/local

Mv hadoop-2.6.0-cdh6.7.0/ hadoop2.6.0

Edit the configuration, and the file successfully configures the corresponding path

Vi / etc/profile

Export HADOOP_HOME=/opt/local/hadoop-2.6.0

Export JAVA_HOME=/usr/local/java/jdk1.8.0_65

Export JRE_HOME=$JAVA_HOME/jre

Export CLASSPATH=.:$JAVA_HOME/lib:$JER_HOME/lib:$CLASSPATH

Export PATH=$JAVA_HOME/bin:$JER_HOME/bin:$PATH:$HADOOP_HOME/bin

Put the configuration file into effect

Source / etc/profile

Java-version to check whether your version is configured successfully.

Configure the address to bind your own hostname and ip address to the firewall

Turn off the firewall on your machine

Chkconfig iptables-list to see which ones are not closed

Chkconfig iptables off shuts down

Modify the domain name of your machine

Vi / etc/sysconfig/network

HOSTNAME=hadoop001

Reboot restart takes effect

Hostsname to see if the hostname has been modified successfully

Vi / etc/hosts

192.168.225.100 hadoop001

Reboot, restart the machine.

2 configure hadoop to be distributed

1 cd / opt/local/hadoop2.6.0/etc/hadoop

Vi vi core-site.xml core file address

Fs.defaultFS

Hdfs:// hadoop001:9000

Hadoop.tmp.dir

/ home/hadoop/app/tmp

2 create the directory Mkdir / home/hadoop/app/tmp

3 vi hdfs-site.xml

Dfs.replication

one

4 jdk cannot be found if vi hadoop-env.sh is not configured to start

Export JAVA_HOME=/usr/local/java/jdk1.8.0_45

5 [root@hadoop001 hadoop] # cp mapred-site.xml.template mapred-site.xml

[root@hadoop001 hadoop] # vi mapred-site.xml

Mapreduce.framework.name

Yarn

6 vi vi yarn-site.xml

Yarn.nodemanager.aux-services

Mapreduce_shuffle

7 vi vi slaves adds the host name to represent the datanode node

It's configured and formatted.

Bin/hdfs namenode-format

Configure ssh password-free login

Cd .ssh

Enter ssh-keygen to enter 4 times

-rw-. 1 root root 1675 Feb 25 05:03 id_rsa

-rw-r--r--. 1 root root 396 Feb 25 05:03 id_rsa.pub

-rw-r--r--. 1 root root 798 Feb 25 05:02 known_hosts

Cat id_rsa.pub > > authorized_keys

Chmod 600 authorized_keys

Ssh + ip or domain name requires a password for the first time, but not for the second time.

Start sbin/start-dfs.sh

The Jpd command appears

8068 NameNode

8373 SecondaryNameNode

8551 ResourceManager

8649 NodeManager

8206 DataNode

24174 Jps

At this point, I believe that everyone on the "hadoop cdh2.6.0-5.7.0 pseudo-distributed version how to build" have a deeper understanding, might as well to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.