Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

Hdfs web_ui in-depth explanation, service startup log analysis, NN SNN relationship

Shulou Source: shulou.com Published: 2022-06-03 07:17:43 11月23日 Update

Hdfs startup sequence:

1 、 namenode

2 、 datenode

3 、 secondnamenode

4 、 yarn

Webui:

Default is port 50070, dfsheath

Ui content explanation:

ClusterID: generated when namenode is formatted (hdfs namenode-format)

BlockpoolID: block pool

For federation and memory management use. )

Namenode Journal status:

It's the edit log.

Namenode storage:

Image file storage information

Launch the log file directory $HADOOP_HOME/logs:

X.log: log information for most applications through log4j logging

X.out records standard output and standard error logs, with a small number of records

Naming rules for log files:

[frame name-user name-process name-hostname-log format suffix]

Secondary namenode:

1. The client's operation on hdfs will be recorded in the edits file, that is, journal.

2. The content in namenode memory is fsp_w_picpath+edits.

3. After namenode's fsp_w_picpath and edite files are merged regularly, they are synchronized to secondary namenode.

Tags: Log file information memory content standard format host name suffix most client that is application data framework user user name directory Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno Shulou Tech Info OPPO Reno Huawei NVidia Docker