- Error on installing Berkshelf 3.0.1+ with OpsWorks on Ubuntu box
- Local OpsWorks testing with Vagrant, Berkshelf 3.0 and Chef 11.10
This article refers to the old revision of storm-docker project. So I modified the description regarding the project cloning.
With storm-docker script, you can setup Apache Storm easily. I describe here how to install Apache Storm and how to deploy a topology on the running containers.
I also forked the wurstmeister's project and customized it (found here). I will create a new post at another time.
Docker 1.0.1 installation
At first, install Docker. Read the page for the instllation instruction.
As described above page, OS X and Windows requires Boot2docker, a tiny Linux VM for Docker, which includes Docker as well.
host $ boot2docker init
This may take a couple of minutes or more as an ISO image is downloaded to
After the command completes, run the following command to start a new image.
host $ boot2docker up
Then for OSX users run:
host $ boot2docker ssh
or for Windows users, use putty or your favorite SSH client instead of the above command line to ssh the boot2docker instance.
boot2docker $ tce-load -wi bash.tcz
Cloning a docker script
Now you can clone storm-docker with
git command, which is installed by default.
boot2docker $ git clone https://github.com/wurstmeister/storm-docker.git boot2docker $ cd storm-docker boot2docker $ git reset --hard 63fa338d9c5f6bc05ae63730d0101a5535ac9081
In order to preserve these changes, exit the ssh session.
boot2docker $ exit
And run the command on the host.
host $ boot2docker save
After saving is finished, the instance is down. Use
boot2docker up to re-launch the image.
Starting Apache Storm
Start the boot2docker isntance.
host $ boot2docker up
SSH to the instance.
host $ boot2docker ssh
Or use Windows SSH client to connect to the instance.
boot2docker $ cd ~/storm-docker boot2docker $ ./start-storm.sh
This will download dependent containers if they aren't yet stored locally.
When the command is completed, you can check the running docker containers.
boot2docker $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 889d11b18998 wurstmeister/storm-ui:latest /bin/sh -c /usr/bin/ 5 seconds ago Up 4 seconds 0.0.0.0:49080->8080/tcp ui 63dcd1ef7e85 wurstmeister/storm-supervisor:latest /bin/sh -c /usr/bin/ 5 seconds ago Up 4 seconds 6700/tcp, 6701/tcp, 6702/tcp, 6703/tcp, 0.0.0.0:49000->8000/tcp supervisor d4076c5fa1a5 wurstmeister/storm-nimbus:latest /bin/sh -c /usr/bin/ 6 seconds ago Up 5 seconds 0.0.0.0:49627->6627/tcp, 0.0.0.0:49772->3772/tcp, 0.0.0.0:49773->3773/tcp nimbus,supervisor/nimbus,ui/nimbus 8a6da03bafcf jplock/zookeeper:latest /opt/zookeeper-3.4.5 41 minutes ago Up 41 minutes 2888/tcp, 3888/tcp, 0.0.0.0:49181->2181/tcp nimbus/zk,supervisor/nimbus/zk,supervisor/zk,ui/nimbus/zk,ui/zk,zookeeper
OK, now Apache Storm is ready!
The started containers are running on Host-only network. As shown above, some of the container ports are forwarded to the boot2docker instance. You can access to the boot2docker port with its IP address from your guest OS.
The boot2docker IP address is displayed by running the following command.
boot2docker $ boot2docker ip
In most cases, the output will be as follows:
The VM's Host only interface IP address is: 192.168.59.103
So you can open the following URL with your browser:
This will render the Storm UI page.
You can get more information regarding the container port forwarding here.
In order to stop the containers, run the following command.
boot2docker $ cd ~/storm-docker boot2docker $ ./destroy-storm.sh
How to deploy and undeploy topologies
In order to deploy your topology, you need to install Apache Storm on your host machine (untar/unzip the archive).
Then set PATH so that you can run
storm command from a terminal.
Deploying a topology
Run the following command on your HOST machine after creating a jar file for your topology.
your-host-machine $ storm jar path/to/your/topology.jar fqdn.of.your.TopologyMainClass \ -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627
Undeploying a topology
Run the following command on your HOST machine and you can remove the deployed topology.
You can specify the waiting time with -w before Storm shuts down the topology.
-w 1 means Storm will wait 1 second before shutting down the topology.
your-host-machine $ storm kill YourTopologyName -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627 -w 1
comments powered by Disqus