What is the structure of Storm in Zookeeper
What is the structure of Storm in Zookeeper? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.
Preservation of Storm in Zookeeper
Brief introduction:
We know that all the state information of Twitter Storm is stored in Zookeeper, and in the previous article on kafka, we know that kafka also stores information in Zookeeper, as well as other similar components: such as double,hbase, etc. There is a dependency on Zookeeper.
Details:
Nimbus assigns tasks by writing status information on Zookeeper, and Supervisor,task gets its own task by reading status in Zookeeper. At the same time, Nimbus is constantly reading the heartbeat signal sent by Supervisor,task to nimbus, so that it can accurately monitor the status of the cluster. As a result, you can restart some "dead" tasks.
The specific source code is backtype.storm.cluster
/-{storm-zk-root}-- the root of storm on zookeeper | Directory |-/ assignments-- the task assignment information of topology |-/ {topology-id}-- the assignments of each | topology is saved below this | The information includes: the corresponding | Code directory on nimbus All | start time of task | | every task and machine, Port mapping | |-/ tasks-- all task |-/ {topology-id}-- the id under this directory is | | {topology-id} topology | | all corresponding task-id |-/ {task-id} | -- what is saved in this file is this | component-id for task: | it may be spout-id or bolt-id | |-/ storms-- this directory holds all the running id of topology | | |-/ {topology-id}-- this file holds some information about this topology | | Including the | name of topology When topology starts running | interval and the status of this topology | (see StormBase class for details) | | / supervisors-- this directory stores all the supervisor | | heartbeat information |-/ {supervisor-id}-- this The heartbeat information of the files saved in supervisor | includes: heartbeat time | Master | Machine name | The running time of the port number of worker | on this supervisor | (see SupervisorInfo class for details) | |-/ taskbeats-- the heartbeat of all task |-/ {topology-id}-- this directory holds the place of this topology | Heartbeat information of some task |-/ {task-id}-- heartbeat information of task Including heartbeat time | interval Task running time and some statistics | Information | |-/ taskerrors-- error information generated by all task | |-/ {topology-id}-- this directory stores the error message under this topology | error message for each task | -- / {task-id}-- after reading the error message of this task, read the above Have you mastered how Storm is structured in Zookeeper? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!
