In late 2009, Amazon introduced a new way for AWS customers to purchase its Elastic Compute Cloud (EC2) service: spot instances.

Instead of buying an instance outright at a fixed price, the price of a spot instance is determined by supply and demand.

So long as your bid for the instance is above the current spot price, you have a fully functional Amazon EC2 instance at your disposal.

For Amazon, the introduction of spot instances was an obvious solution to a costly problem: unused capacity. After all, Amazon has built one of the world's largest cloud infrastructures, and if segments of it sit idle, Amazon effectively loses money.

For savvy AWS customers, this created an enticing opportunity. If the spot price for instances is typically below the price for on demand and reserved instances, it's possible to lower the costs of using EC2, in some instances significantly.

There's one caveat, of course: if the spot price frequently exceeds your maximum bid for spot instances, you'll find yourself instance-less. When that happens, any processes running on spot instances are dead, leaving you in a bad situation, particularly if you're running critical and customer-facing services.

Fortunately, there historically hasn't been huge volatility in the market for spot instances, but when there is, it can be painful. Just ask SEOmoz. The popular provider of web-based SEO software is a heavy user of Amazon AWS, and relies on spot instances to reduce its costs.

The economic advantages aren't hard to understand. As SEOmoz's Brandon Forehand explains on the company's developer blog, SEOmoz typically pays about 24 cents per hour for an instance that would cost 68 cents per hour as an on-demand instance. That adds up.

But recently, spikes in the spot prices of instances have caused major problems for SEOmoz:

However, we’ve been noticing quite a bit of pain the last week due to some extreme volatility in the spot price market. Normally, we use m1.xlarge instances, but over the last two days we’ve consistently lost machines in multiple availability zones due to the spot prices suddenly jumping to ridiculously high amounts.

How big were the spikes? Some took the spot price to a whopping $999 per hour!

According to Brandon, past spikes have typically been measured in minutes, and haven't been anywhere near as extreme as the ones seen in the past week.

Obviously, $999/hour far exceeded SEOmoz's bid for spot instances, so even though "we’ve always priced in a certain amount of volatility", the company was left powerless as its spot instances were pushed offline, creating outages for customers.

Whether SEOmoz's recent experience is a harbinger of things to come remains to be seen, but it does provide an important lesson for other companies using spot instances. The lesson: when you turn cloud infrastructure into a commodity that's bought and sold, you have to recognize that there is downside exposure.

Playing in the market for spot instances is, in many respects, not that dissimilar from playing in the stock and commodities markets.

One strong move against your position can wipe out all your gains, so investing most or all of your customer-facing services in spot instances is probably not a good idea, even if appears to be when things are going your way.

Instead, a more prudent investment strategy calls for using spot instances more sparinly, ensuring that they don't make too much of your 'portfolio', and that if you lose them, your entire portfolio doesn't blow up.

Patricio Robles

Published 28 September, 2011 by Patricio Robles

Patricio Robles is a tech reporter at Econsultancy. Follow him on Twitter.

2646 more posts from this author

You might be interested in

Comments (1)



Use a service for a production system that can be terminated capriciously without your control?


Where else is SEOmoz cutting corners if they know they could lose their instances at any moment?

There's "learning a lesson" and then there's reading the documentation.

almost 7 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.