Apache Storm environment configuration
i. Prepare for
Apache-storm-0.9.3 zookeeper-3.4.6
ii. structure
One server with nimbus and zookeeper, and two supervisor.
iii. Configuration
ZooKeeper:
# vi conf/zoo.cfg
ClientPort=10205server.0=nimbus:2888:3888
# vi / var/zookeeper/myid
Storm:
# vi conf/storm.yaml
Storm.zookeeper.servers:-"nimbus"-"supervisor1"-"supervisor2"-"supervisor3" storm.zookeeper.port: 10205storm.local.dir: "/ mnt/storm" # indicates the local directory that storm needs to use, such as saving the jar file of topology, and java.library.path: / usr/local/lib:/opt/local/lib:/usr/libnimbus.host: "nimbus" supervisor.slots.ports: # indicates the number of slots of supervisor node That is, each supervisor can open several worker processes. -6700-6701-6702-6703
iv. Steps
1. Start zookeeper on nimbus. Zk must be started first, otherwise nimbus startup will report an error.
#. / zkServer.sh startJMX enabled by defaultUsing config: / root/storm/zookeeper-3.4.6/bin/../conf/zoo.cfgStarting zookeeper... Whether the STARTED# bin/zkCli.sh stat / test service starts normally Connecting to localhost:21812015-05-28 05Partition 32 myid 11600 [myid:]-INFO [main:Environment@100]-Client environment:zookeeper.version=3.4.6-1569965 Built on 02 GMT2015 20 Accord 2014 09:09 GMT2015-05-28 05 INFO 32 INFO 11604 [myid:]-INFO [main:Environment@100]-Client environment:host.name=mats03.webex.com2015-05-28 05 05 32 Swiss 11605 [myid:]-INFO [main:Environment@100]-Client environment:java.version=1.7.0_452015-05-28 05 05 INFO [main:Environment@100]-Client environment:java.vendor=Oracle Corporation2015-05 -28 05 myid 32 Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre2015 11607 [myid:]-INFO [main:Environment@100]-Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre2015-05-28 05 15 15 22 22 22 [myid:]-INFO [main:Environment@100]-Client environment:java.class.path=/root/storm/zookeeper-3.4.6/bin/../build/classes:/root/storm / zookeeper-3.4.6/bin/../build/lib/*.jar:/root/storm/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/storm/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/root/storm/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/root / storm/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/root/storm/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/root/storm/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/root/storm/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/root/storm/zookeeper -3.4.6/bin/../conf::/home/oracle/OraHome1/jlib:/home/oracle/OraHome1/rdbms/jlib2015-05-28 05Participity11607 [myid:]-INFO [main:Environment@100]-Client environment:java.library.path=:/home/oracle/OraHome1/lib:/opt/webex/mats/bin:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib2015-05-28 05:32 11608 [myid:]-INFO [main:Environment@100]-Client environment:java.io.tmpdir=/tmp2015-05-28 05PUBG 32VAND 11608 [myid:]-INFO [main:Environment@100]-Client environment:java.compiler=2015-05-2805PUBG 32VOUR 11608 [myid:]-INFO [main:Environment@100]-Client environment:os.name=Linux2015-05-28 05VOUTHANG 11608 [myid:]-INFO [main:Environment@100]-Client Environment:os.arch=amd642015-05-28 05 INFO 32 INFO 11608 [myid:]-INFO [main:Environment@100]-Client environment:os.version=2.6.32-279.9.1.el6.x86_642015-05-28 05 05 32 INFO 11608 [Client environment:user.name=root2015:]-INFO [main:Environment@100]-Client environment:user.home=/root2015- 05-28 05 myid 32 main:Environment@100 11609 [myid:]-INFO [main:Environment@100]-Client environment:user.dir=/root/storm/zookeeper-3.4.62015-05-28 05 05 Fringe 32 Swiss 11610 [myid:]-INFO [main:ZooKeeper@438]-Initiating client connection ConnectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@75ab5e0b
two。 Start the primary node nimbus and ui
Nohup bin/storm nimbus & nohup bin/storm ui &
3. Start supervisor from the slave node
Nohup bin/storm supervisor &
4. View interface: http://:18080/index.html
V. Example
This is the example provided in the storm installation package, the path apache-storm-0.9.3\ examples\ storm-starter\ src\ jvm\ storm\ starter, and submit a topology named wordcountexample.
/ storm jar / home/slimttyagent/storm-starter-topologies-0.9.3.jar storm/starter/WordCountTopology wordcountexample519 [main] INFO backtype.storm.StormSubmitter-Jar not uploaded to master yet. Submitting jar...525 [main] INFO backtype.storm.StormSubmitter-Uploading topology jar / home/slimttyagent/storm-starter-topologies-0.9.3.jar to assigned location: / var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jarStart uploading file'/ home/slimttyagent/storm-starter-topologies-0.9.3.jar' to'/ var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444fMurb86b0e5a45b99da.jar' (3248859 bytes) [=] 3248859 / 3248859File'/ home/slimttyagent/storm-starter-topologies-0.9.3.jar' uploaded to'/ var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f main b0e5a45b99da.jar' (3248859 bytes) 557 [main] INFO backtype.storm.StormSubmitter-Successfully uploaded topology jar to assigned location: / var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jar558 [main] INFO backtype.storm.StormSubmitter-Submitting topology wordcountexample in distributed mode with conf {"topology.workers": 3 "topology.debug": true} 737 [main] INFO backtype.storm.StormSubmitter-Finished submitting topology: wordcountexample
