Wednesday, October 9, 2013

Apache Whirr with

I've been recently playing with Apache Whirr to spin up clusters and wanted to try and write some of my own services - but didn't want to pay for testing against Amazon EC2 (I've expired my 1 year / 750 hour trial period and Amazon charges by the rounded up hour).

Whirr supports a number of providers (under the hood Whirr uses JCloud, so anything that JCloud can interface with), and I've started to look into using OpenStack. You can check out DevStack (a single node OpenStack install) and run on your own system (assuming you have CPU hardware virtualization, enough memory etc) but another option is to sign up to TryStack.

TryStack is a free testing sandbox for OpenStack apps - signup on their web page to get a free account (you'll need a Facebook account to get on the x86 cluster). Once you have an account approved, you can log on to the x86 cluster. From there you can browse the images available and configure your default security group (much like you would on EC2).

To test, I thought I'd spin up a single ZooKeeper instance. As WHIRR takes care of the security groups for you, you won't need to this manually. You will need to create your own Key Pair (ssh-keygen -t rsa -P '' -f id_rsa_trystack)

Finally you'll need to generate an API Password Key. This is under the Settings link (top right), and the select the API Password link from the left menu. Click the Request API Password button and note the generated password. Now you'll need to play the waiting game as the passwords are sync'd once an hour - if you keep refreshing the page every 10 minutes, eventually the displayed password will disappear, denoting that it's in sync.

Now you can finally use Whirr to spin up that ZooKeeper instance. Download Apache Whirr and unpack to a directory. You'll need to create a configuration file which will store the cluster configuration information ($WHIRR_HOME/conf/trystack-zookeeper.conf for example):


Finally we can now use the whirr launch-cluster command to bring up a ZooKeeper node on TryStack:

Of course, i didn't get to the magical set of arguments straight off the bat, and encountered several error messages along the way.
  • Security credentials were the first problem, i didn't realize that your user name and tenantName were the same (hence the whirr.identity value being the same, delimited by a single colon). I had to fall back to the OpenStack Web API to validate i had the right credentials:
  • Image Id's were the next problem - not obviously documented anywhere, but the image id must be prefixed with the region or zone, in this case the magical "RegionOne" prefix
  • Lastly the location-id is again the "RegionOne" string (see WHIRR-728)