SLTechnology News&Howtos › Internet Technology ›

Hadoop source code directory tree

Shulou Source: shulou.com Published: 2022-06-03 05:56:57 10月02日 Update

1 、 NameNode

-FSNamesystem:

-FSDirectory directory tree

-INodeDirectory node directory

-List children child node)

-INodeDirectory

-INodeFile

-BlockInfo [] file block (one file block includes all DataNode storage information)

-Object [] triplets storage (DatanodeStorageInfo packet message {DatanodeDescriptor,DatanodeStorage})

-BlockCollection saves INodeFile

-INodeSymlink

-INodeMap

-BlockManager hdfs Block Manager

-DatanodeManager (register, delete, add)

-HostFileManager reads include/exclude files, fine management DataNode on the shelf, off the shelf

-NetworkTopology network topology diagram for rack awareness

-HeartbeatManager heartbeat package manager to maintain communication with DataNode

-BlockTokenSecretManager Block Security Authentication Manager

-BlocksMap block mapping [BlockInfo] stores information for all blocks

-LeaseManager Lease Manager

-all RPC API calls provided by NameNodeRpcServer

-web access provided by NameNodeHttpServer

-HAContext {NameNodeHAContext} for NameNodeHA high availability

-HAState includes three states: ActiveState, BackupState and StandbyState

2 、 DataNode

-BlockPoolManager

-List

-BPServiceActor registration, heartbeat, receive block, delete block, update block, bad block report, etc.

-FsDatasetSpi {FsDatasetImpl} data node, operating on all directory block files

-DataXceiverServer data node, data receiving and sending service

-DataBlockScanner data node block scanner

-DirectoryScanner data node directory scanner

-BlockPoolTokenSecretManager

-Web access provided by HttpServer2