SLTechnology News&Howtos › Internet Technology ›

Big data Environment Building-hadoop 2.x distributed deployment-Cluster configuration

Shulou Source: shulou.com Published: 2022-06-03 07:24:04 09月23日 Update

1. Download hadoop and decompress it

2. Assign roles to hadoop files

Chown-R hadoop:hadoop hadoop

Group name: role name file name

3. Ls-l to check whether the role assignment is successful or not

4. Create the file directory required by hadoop

Mkdir-p / data/dfs/name

Mkdir-p / data/dfs/data

Mkdir-p / data/tmp

5. Assign role permissions to your own user directories, such as hadoop users

Chown-R hadoop:hadoop hadoop/data

6. Ls-l to check whether the allocation is successful.

7. Modify hadoop configuration file

A 、 core-site.xml

B 、 hdfs-site.xml

C 、 mapred-site.xml

D 、 slave

8. Configure hadoop environment variables

HADOOP_HOME=/usr/java/hadoop

PATH=$HADOOP_HOME/bin:$PATH

Export HADOOP_HOME PATH

9. Format hadoop bin/hadoop namenode-format

10. Start hadoop sbin/start-all.sh

11. Jps to check whether the process is started

NodeManager

ResourceManager

Jps

DataNode

NameNode

SecondaryNameNode

12. Hdfs creates a directory

Bin/hadoop fs-mkdir / dajiangtai

13. Add files to the dajiangtai directory of hdfs

Hadoop fs-put / usr/java/hadoop/djt.txt / dajiangtai

14. Verify address

Http://cjqlinux:8088/cluster/apps

Http://cjqlinux:50070