Common ports of Hadoop 2.x and their viewing methods
I. Common ports
Component node default port configuration purpose description HDFSDataNode50010dfs.datanode.addressdatanode service port, port for data transmission HDFSDataNode50075dfs.datanode.http.addresshttp service port HDFSDataNode50020dfs.datanode.ipc.addressipc service port HDFSNameNode50070dfs.namenode.http-addresshttp service port HDFSNameNode50470dfs.namenode.https-addresshttps service port HDFSNameNode8020fs.defaultFS receives the Client connection RPC port, which is used to obtain file system metadata information. HDFSjournalnode8485dfs.journalnode.rpc-addressRPC Service HDFSjournalnode8480dfs.journalnode.http-addressHTTP Service HDFSZKFC8019dfs.ha.zkfc.portZooKeeper FailoverController Applications manager (ASM) port YARNResourceManager8030yarn.resourcemanager.scheduler.addressscheduler component for NN HAYARNResourceManager8032yarn.resourcemanager.addressRM IPC port YARNResourceManager8031yarn.resourcemanager.resource-tracker.addressIPCYARNResourceManager8033yarn.resourcemanager.admin.addressIPCYARNResourceManager8088yarn.resourcemanager.webapp.addresshttp service port YARNNodeManager8040yarn.nodemanager.localizer.addresslocalizer IPCYARNNodeManager8042yarn.nodemanager.webapp.addresshttp service port container manager port in YARNNodeManager8041yarn.nodemanager.addressNM YARNJobHistory Server10020mapreduce.jobhistory.addressIPCYARNJobHistory Server19888mapreduce.jobhistory.webapp.addresshttp service port HBaseMaster60000hbase.master.portIPCHBaseMaster60010hbase.master.info.porthttp service port HBaseRegionServer60020hbase.regionserver.portIPCHBaseRegionServer60030hbase .regionserver.info.porthttp service port HBaseHQuorumPeer2181hbase.zookeeper.property.clientPortHBase-managed ZK mode Using a separate ZooKeeper cluster does not enable the port. HBaseHQuorumPeer2888hbase.zookeeper.peerportHBase-managed ZK mode, using a separate ZooKeeper cluster will not enable this port. HBaseHQuorumPeer3888hbase.zookeeper.leaderportHBase-managed ZK mode, using a separate ZooKeeper cluster will not enable this port. Export PORT= in HiveMetastore9083/etc/default/hive-metastore to update the default port
Export HIVE_SERVER2_THRIFT_PORT= in HiveHiveServer10000/etc/hive/conf/hive-env.sh to update the default port
In ZooKeeperServer2181/etc/zookeeper/conf/zoo.cfg, the port where clientPort= provides services to clients is server.x= [hostname]: nnnnn [: nnnnn] in ZooKeeperServer2888/etc/zookeeper/conf/zoo.cfg. The blue part of follower is used to connect to leader, and only listen on this port on leader. Server.x= [hostname]: nnnnn [: nnnnn] in ZooKeeperServer3888/etc/zookeeper/conf/zoo.cfg, the blue part is used for leader election. Only required if the electionAlg is 1 and 2 or 3 (the default).
All port protocols are based on TCP.
For more information on some ports, please see: http://blog.csdn.net/xiaolang85/article/details/8647602
2. Viewing method
For NameNode and DataNode, you can view their information through the following Web page:
I. NameNode: http://{NameNodeServer}:50070/
/ dfshealth.jsp HDFS information page with links to view the file system
/ dfsnodelist.jsp?whatNodes= (DEAD | LIVE) datanode that displays the status of DEAD or LIVE
/ fsck runs the fsck command, which is not recommended when the cluster is busy!
II. DataNode: http://{DataNodeServer}:50075/
/ blockScannerReport each datanode specifies an interval to verify block information
iii. For both NameNode and DataNode, the following Web URL exists:
/ logs log file list for downloading and viewing
/ logLevel allows you to set the logging level of log4j, similar to hadoop daemonlog
/ stacks the stack trace of all threads, which is very helpful for debug
/ Metrics of the jmx server, output in JSON format.
/ jmx?qry=Hadoop:* returns all hadoop-related metrics.
The / jmx?get=MXBeanName::AttributeName query specifies the value of the property specified by bean, for example, / jmx?get=Hadoop:service=NameNode,name=NameNodeInfo::ClusterId returns ClusterId. The processing class for this request: org.apache.hadoop.jmx.JMXJsonServlet
3. Configure Eclipse
1. In the Define Hadoop Location interface, for pseudo-distributed deployments with default configuration:
Map/Reduce (V2) Master: localhost:50020
DFS Master: localhost:8020
