Shulou(Shulou.com)06/03 Report--

Hadoop3 cluster mode includes independent mode, pseudo-distributed mode and fully distributed mode.

1 independent mode

Installation

A) download jdk-8u65-linux-x64.tar.gz

B) tar on

$> su centos; cd ~

$> mkdir downloads

$> cp / mnt/hdfs/downloads/bigdata/jdk-8u65-linux-x64.tar.gz ~ / downlooads

$> tar-xzvf jdk-8u65-linux-x64.tar.gz

C) create / soft folder

$> sudo mkdir / soft

$> sudo chown centos:centos / soft

D) move the files opened by tar to / soft

$> mv ~ / downloads/jdk-1.8.0_65 / soft/

E) create symbolic links

$> ln-s / soft/jdk-1.8.0_65 / soft/jdk

F) verify that the jdk installation was successful

$> cd / soft/jdk/bin

$>. / java-version

Configure jdk environment variables

1. Edit / etc/profile

$> sudo nano / etc/profile

...

Export JAVA_HOME=/soft/jdk

Exprot PATH=$PATH:$JAVA_HOME/bin

two。 Make the environment variable effective immediately

$> source / etc/profile

3. Go to any directory and test whether it is ok

$> cd ~

$> java-version

Pay attention

The folder you create must be the same as the users you log in.

Install HADOOP softwar

1. Install hadoop

A) download hadoop-2.7.3.tar.gz

B) tar on

$> su centos; cd ~

$> cp / mnt/hdfs/downloads/bigdata/hadoop-2.7.3.tar.gz ~ / downloads

$> tar-xzvf hadoop-2.7.3.tar.gz

C) none

D) move the files opened by tar to / soft

$> mv ~ / downloads/hadoop-2.7.3 / soft/

E) create symbolic links

$> ln-s / soft/hadoop-2.7.3 / soft/hadoop

F) verify that the hadoop installation was successful

$> cd / soft/hadoop/bin

$>. / hadoop version

two。 Configure hadoop environment variables

$> sudo nano / etc/profile

...

Export JAVA_HOME=/soft/jdk

Exprot PATH=$PATH:$JAVA_HOME/bin

Export HADOOP_HOME=/soft/hadoop

Export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

3. Take effect

$> source / etc/profile

4. Go to any directory and test whether it is ok

$> cd ~

$> hadoop version

5.nothing!

There is no need to enable a separate hadoop process.

6.hafs dfs-ls / home has the same effect as ls in linux

Pseudo distribution pattern (Pseudodistributed mode)

A) enter the ${HADOOP_HOME} / etc/hadoop directory

Three modes coexist

1. Create three configuration directories, which are equivalent to the hadoop directory

${hadoop_home} / etc/local

${hadoop_home} / etc/pesudo

${hadoop_home} / etc/full

two。 Create a symbolic connection

$> ln-spesudo hadoop

B) Edit core-site.xml

Fs.defaultFS

Hdfs://localhost/

C) Edit hdfs-site.xml

Dfs.replication

one

D) Edit mapred-site.xml

Note: cp mapred-site.xml.template mapred-site.xml

Mapreduce.framework.name

Yarn

E) Edit yarn-site.xml

Yarn.resourcemanager.hostname

Localhost

Yarn.nodemanager.aux-services

Mapreduce_shuffle

Configuration: core-site.xml hdfs-site.xmls yarn-site.xml mapred-site.xml

F) configure SSH

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.