ZeroMQ Musings and Server Build

I just read an excellent writeup about ZeroMQ (ØMQ/ZMQ) yesterday on igvita.com.  This software appears to have been around a while but I hadn't seen it before.  It's really quite impressive.  So, I found myself quite curious to play around with it a bit this weekend.  So, I built a little rig that would let me do that based on Ubuntu 10.04 LTE.

I wanted to use the ruby bindings for my playing around and ruby 1.9.2p0.  I quickly found that most of the easy to find examples out there are in C or Python.  But, there is still some good stuff.  I'll add some of things I found as links at the bottom of this post. 

The server build instructions  here in case anyone else was interested.  The following steps will yield you a basic build with which you may test ZMQ w/ by writing ruby code.

If anyone has thoughts, ideas or improvements on this setup by all means please do let me know!  Comments have been off for a while on my blog but I'll be turning them back on after this post.

Server Build - Ruby 1.9.2p0 + ZMQ + Ruby Bindings

While playing around a bit this weekend with zeroMQ and wanting to mess w/ the ruby bindings I found I needed to build a server.  It wasn’t difficult but these are the steps which might help you get going quickly on the rackspace cloud.

Provision Your Server

I grabbed mine from the Rackspace cloud.  Your milage may vary but I know that a RS 10.04 is a well build no frills ubuntu server.  I really like using their templates as the basis for my builds.  Once you have your server up and you are logged in:

You are now all set with a ruby 1.9.2p0 and zeroMQ enabled server on Ubuntu 10.04 in the Rackspace cloud.  If this was helpful then let me know what you do with it as this is a very exciting combination.

Note:  This will work well with any Ubunutu 10.04 server. It doesn’t  have to only be a Rackspace Cloud Server.

For next steps take a look at the basic zeroMQ example published by Will’s Web Miscellany.

Of other notable interest is the Mongrel2 project which incorporates ZMQ.  The mongrel2 manual is very good reading as well.

Other helpful Links I found have been tagged on my Delicious acct here.  I'll be adding more as well as I find them.

Some Cloud Thoughts on a Clear and Sunny Day

Cloud Computing is a deployment model and cloud computing is a business model.  Cloud computing is not some silver bullet magical thing.  It's not even easy *gasp* sometimes.

As a deployment model cloud computing can it is simply summed up as on-demand, self-service, reliable, and low to no capital costs services for the consumer.

As a business model it is summed up as, again, low to no long term capital costs (and the associated depreciation) and pay as you go service provider pricing models.  In reality these are mountains of micro transactions aggregated into monthly and yearly billing cycles.  For example, I spent $0.015 for a small compute instance w/ a cloud infrastructure provider because I just needed an hour of an Ubuntu 10.04 linux machine to test a quick software install combination and update a piece of documentation.  I'll get a bill for that at the end of the month.  Get this...

An hour of compute time costs me 3.3 times LESS than a piece of hubba bubba chewing gum cost me at $0.05 (one time use only) over 30 years ago. #cloud

Enterprises and service providers are learning very quickly from the how the early public cloud vendors how to do things differently and often more efficiently.  It was well summed up in the Federal CTO's announcement of the government application cloud.  Basically, that we saw that consumers could get IT services for orders of magnitude less than we could.  So, we're fixing that by emulating what the companies that service the consumers are doing. Smart.  Bechtel did this exact same thing years ago when analyzing that the cost per GB of storage for Amazon was orders of magnitude less than Bechtel could and asked the very important question why and then answered it very well.

A couple of years ago now I helped found a company called nScaled.   nScaled does, business continuity as a service.  It is only possible with the resources, at the price, and at the speed we have moved because of following cloud computing deployment and business models.  It would not have been possible for us to build this business when we did and the way we have without these models.  

In March 2008 I called cloud computing a renaissance.

It is my opinion that Cloud Computing is a technology architecture evolution that, when properly applied to business problems, can enable a business revolution. I've been saying this for a while but in recent weeks I have actually come to prefer the term renaissance over revolution.

Today, two years into a startup that uses the raw power of cloud computing deployment and business models across the board to enable new ways for companies to consume disaster recovery and business continuity solutions I can say without a doubt that I believe that cloud computing is a renaissance more than ever before!

Excellent RailsEnvy List in Ep.101

RailsEnvy Episode #101

The list of links from this podcast episode was particularly intriguing for me this week.

Of particular interest to me is:

TorqueBox - JRuby backed Rails application platform (and more) build on JBoss AS.  Very intriguing and I'll be experimenting right away with this for an application I've just pushed out into production.

ShardTheLove - An Active Record horizontal sharding solution with build in support for migrations, testing, and more.  I will also be evaluating this for inclusion into a new application I am just launching.

Jammit - A static asset packing solution for Rails applications. Finding a good solution for this can sometimes be challenging.  However, doing it in any modern web application is pretty much mandatory.  I look forward to testing this library.

Great stuff and worth a look if your pumping out Rails applications that you want to be scalable on-demand.

Monitoring – Updated and Revisited

I’ve written several times on this blog about various monitoring tools and services.  But, in light of a recent project I’ve been working on for the last few months I have some updates.

The Overall Monitoring Architecture

There are areas of monitoring that I usually like to pay close attention to with a live web application.

1. Process Monitoring - This makes sure things are running and stay running within certain tolerances. Examples are God, Monit, SMF.  Your choice will depending on your operating system and preferences with scripting languages.
2. Resource Monitoring - This is fine grained CPU, Memory, Disk Space, Disk IO, Networking, application server threads, and much more. Examples are Nagios, Ganglia, and Munin. Choosing correctly depends on your specific situation.  There is a worth newcomer on the block called Reconnoiter that also looks very promising.
3. UpTime Monitoring - This is the only monitor people usually do if they do any at all. This should be a disinterested 3rd party to provide accountability and what I call a 3rd party eye in the sky should any dispute about uptime arise.  I like pingdom and there are even free services as well.  I’ve also been using CloudKick in some situations for this purpose as well.

Those three above are from a post I wrote some time ago.  Today, I’m adding a 4^th item to that list because it has finally become easy enough and reasonably affordable to add now that there is an affordable choice:

4. Synthetic Transaction Monitors – These actually perform tests of processes a user might go through in your application and report back any anomalies if they occur along with an error report, screen shot, and other data as appropriate.  I’ve been using a tool called BrowserMob and Selenium IDE for this.  You create scripts w/ Selenium, upload them to browswermob and then setup a monitor script.  That’s a simplified overview of course but it’s really quite effective and relatively affordable compared to historical solutions for synthetic transaction monitoring.  Historically it was prohibitively expensive to do synthetic transaction monitors.

The Monitoring Tools I use

What follows are some of my personal current favorites to meet the above goals.

Munin > http://munin.projects.linpro.no/

One of the things on my list for a while to get done is enable munin across your systems.  I use it a lot successfully.  You can see a demo here:

Monit > http://mmonit.com/monit/

Pingdom > http://www.pingdom.com

Things I’m testing and have high hopes for are Reconnoiter, BrowserMob monitoring, CloudKick

 

D-I-D Approach to Scalabilty - Article at AKF Partners

One of the blogs I frequent is AKF Partners.  They write some top quality content there.  A recent post introduced what they call the D-I-D approach to scalability.  This stands for Design, Implement, and Deploy.  It advocates planning for scalability.  *GASP*  Say what?!  Plan for scalability?  I kid.. I kid...  This is something that's all to rare and I usually get told that planning for scalability is a waste of time.  I 100% disagree with that attitude and do like that approach AKF is espousing.  I posted some comments to their blog entry and will just dupe those here. Nothing like quoting yourself anyway right?

My comments:

Excellent write up. Thank you. This is almost exactly what I advocate day after day regarding planning for scalability. I like that you’d put a bit of a framework around it.

Personally, I treat scalability concerns the same as any other “feature” in a project in a Agile Development context. It’s brought up and discussed briefly in scrums, possibly handled in a follow up meeting, and then either put on the backlog to be prioritized w/ everything else or worked on next if necessary.

It’s often very, very difficult to get some teams to think pro-actively about scalability AND agree to table it for later. I think this is because sometimes when you have these discussions people realize that they need to “fix” something and can’t really help themselves.

Of course, these days with cloud computing gaining so much traction and resources being available on-demand more than ever at a moments notice it’s getting easier. But, if you don’t architect for scalability at the outset you may find when you get the implementation and deployment phases that it’s impossible to do what needs to be done. But, that’s a whole other topic I suppose.

Cheers!