Two operating modes of Storm
Local mode operation
The local mode is similar to the storm cluster, which is a process used to write and test topology. Running topology in local mode is similar to running topology on a cluster. Create a local cluster:
Import backtype.storm.LocalCluster
LocalCluster cluster = new LocalCluster ()
Submit the cluster to use submitTopology
Kill the cluster using killTopology
Shut down a local cluster using cluster.shutdown ()
Public configuration in local mode:
Config.TOPOLOGY_MAX_TASK_PARALLELISM
Config.TOPOLOGY_DEBUG:
Production cluster operation
I. steps
1. Define topology. If you use Java, use TopologyBuilder to define it.
2. Use StormSubmitter to submit topology to the cluster. The required parameters need topology name, parameter configuration of topology, and topology itself.
Example:
Config conf = new Config ()
Conf.setNumWorkers (20)
Conf.setMaxSpoutPending (5000)
StormSubmitter.submitTopology ("mytopology", conf, topology)
3. Use storm client commands to create the jar package, and the jar package contains all your code.
4. Submit topology and specify parameters for jar package
Storm jar path/to/allmycode.jar org.me.MyTopology arg1 arg2 arg3
Example:
1 public static void main (String [] args) throws Exception {
2 TopologyBuilder builder = new TopologyBuilder ()
3 builder.setSpout ("random", new RandomWordSpout (), 2)
4 builder.setBolt ("transfer", new TransferBolt (), 4). ShuffleGrouping ("random")
5 builder.setBolt ("writer", new WriterBolt (), 4) fieldsGrouping ("transfer", new Fields ("word"))
6 Config conf = new Config ()
7 conf.setNumWorkers (4); / / set to start 4 Worker
8 conf.setNumAckers (1); / / set up an ack thread
9 conf.setDebug (true); / / set up to print all sent messages and system messages
10 StormSubmitter.submitTopology ("test", conf, builder.createTopology ())
11}
II. Public allocation
Config.TOPOLOGY_WORKERS: sets the number of worker that executes topology
Config.TOPOLOGY_ACKER_EXECUTORS:
Config.TOPOLOGY_MAX_SPOUT_PENDING
Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS: the default is 30s
Config.TOPOLOGY_SERIALIZATIONS
3. Kill topology
Storm kill stormname
Update a running topology
Kill a running topology, submit a new topology, and plan to use the storm swap command
5. Monitor the work logs of topology using Storm UI or cluster
