Shulou(Shulou.com)06/03 Report--

Author | Bai Song

Note: this article is original, citation reprint should be related to Qilan.

1. Org.apache.giraph.bsp.CentralizedService interface

Function: Basic service interface shared by both CentralizedServiceMaster and CentralizedServiceWorker.

Cdn.xitu.io/2019/7/19/16c08d3d927c9e2a?w=640&h=291&f=png&s=61631 ">

2. Org.apache.giraph.bsp.CentralizedServiceMaster interface

Functions: At most, there will be one active master at a time, but many threads can be trying to be the active master.

3. Org.apache.giraph.bsp.CentralizedServiceWorker interface

Function: All workers should have access to this centralized service to execute the following methods.

4. Org.apache.giraph.bsp.BspService abstract class

Function: Zookeeper-based implementation of CentralizedService.

5. Org.apache.giraph.master.BspServiceMasterclass

Function: ZooKeeper-based implementation of CentralizedServiceMaster.

6. Org.apache.giography .worker.BspServiceWorker class

Function: ZooKeeper-based implementation of CentralizedServiceWorker.

The BspServiceWorker class has WorkerClient and WorkerServer instances, which send data through Netty as the client and server of IPC communication, respectively. The WorkerClient instance is actually a NettyWorkerClient object, and the WorkerServert instance is actually a NettyWorkerServer object.

NettyWorkerClient implements WorkerClient interface, NettyWorkerServer implements WorkerServer interface.

The constructor of the NettyWorkerServer class creates a NettyServer object for the communication of the underlying IPC, and a ServerData object as a data entity. The ServerData contains the partitionStore, edgeStore, incomingMessageStore, currentMessageStore, aggregation values of the Worker, and so on.

Create a NettyClient object in the constructor of the NettyWorkerClient class, which is used for the communication of the underlying IPC as the client.

7. Org.apache.giraph.worker.InputSplitsCallable abstract class, which inherits Callable interface.

Function: used to load vertex or edge input splits, each thread has a WorkerClientRequestProcessor instance (actually a NettyWorkerClientRequestProcessor object), which is responsible for sending data to the remote worker.

The WorkerClient object used by the NettyWorkerClientRequestProcessor object to send is the WorkerClient object in the BspServiceWorker.

The readInputSplit () method in the VertexInputSplitsCallable class is used to read the vertex information from the split, and then call the sendVertexRequest () method of the NettyWorkerClientRequestProcessor object to send the vertex to the Partition to which it belongs.

8. Org.apache.giraph.graph.ComputeCallable class, which inherits Callable interface.

Complete the process of "calculate-communicate-synchronize" in this object. Each thread has an instance of WorkerClientRequestProcessor (actually a NettyWorkerClientRequestProcessor object) that is responsible for sending data to the remote worker.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.