How does hadoop remove a node from a cluster
This article focuses on "how to remove a node from a cluster by hadoop". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how hadoop removes a node from a cluster.
1. First check the Average block replication of the whole cluster. If it is greater than 2, even if you pull out a node directly, you will not lose data.
Hadoop fsck / check the file system status of the cluster
The redundancy multiple of the file can be set manually, and for safe backup, hadoop fs-setrep-w 3-R can be used for critical data.
two。 Add configuration to the following two files and fill in the name of the node to be dismissed in the excludes
Mapred-site.xml
Mapred.hosts
Names a file that contains the list of nodes that may
Connect to the jobtracker. If the value is empty, all hosts are
Permitted.
Mapred.hosts.exclude
HADOOP_HOME/conf/excludes
Names a file that contains the list of hosts that
Should be excluded by the jobtracker. If the value is empty, no
Hosts are excluded.
Hdfs-site.xml
Dfs.hosts
Names a file that contains a list of hosts that are
Permitted to connect to the namenode. The full pathname of the file
Must be specified. If the value is empty, all hosts are
Permitted.
Dfs.hosts.exclude
HADOOP_HOME/conf/excludes
Names a file that contains a list of hosts that are
Not permitted to connect to the namenode. The full pathname of the
File must be specified. If the value is empty, no hosts are
Excluded.
Configure the hostname of the machine in the excludes file.
Run on namenode: hadoop dfsadmin-refreshNodes
Run on jobtracker: hadoop mradmin-refreshNodes
The "hadoop dfsadmin-refreshNodes" triggers the Decommission process, and during the Decommission process, the cluster redundancy the data on the Decommission node to several other points
At this point, I believe you have a deeper understanding of "how hadoop removes a node from the cluster". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!
