Yesterday, we attended the Amazon Web Services Summit in New York where Dr Werner Vogels, CTO, Amazon, gave the keynote speech highlighting how cloud services will transform how we do business.
Though some critics think cloud services may have unforeseen challenges, Vogals somewhat salesy keynote also had representatives of companies using Amazon cloud services come to the stage to say why the cloud is enabling their businesses to do things they could never do before.
As these (and most) businesses are discovering, a data revolution is taking place. The amount of information we need to process, map and store is growing at exponential rates. So in comes cloud services.
According to Vogels, the cloud is not just about saving money and doing things faster. It transforms what is possible. Everything fails all the time so you need to be able to be flexible. Throughout his keynote, Vogels highlighted seven transformations he thinks cloud services will make.
Transformation one: Distributed architectures made easy with high availability
When you are building distributed architectures using traditional infrastructures, it's difficult to do. Single data centers can fail and this means down time for any services on them.
For Amazon web services, their "data centre" is not a building but it is an availability zone. In any given area, you are backed up in multiple places so in cases of outages or natural disasters (or the hardest failures as Vogels called it), your service will still be available at a moments notice through an API.
Transformation two: Embracing the security of shared services
Vogels likened most sites as a castle with a moat to protect it. He argued that a moat is not sufficient as a single breach will immediately expose services. So in order to protect an application, an application has to protect itself.
By having applications in the cloud, you can choose the right security model for each application and keep them separate from one another.
Amazon's number one investment area is security. Vogels rightly stated that you cannot do business on the internet and not think about security.
Transformation three: We're moving from scaling by architecture to scaling by command
Traditionally, if you needed to scale your application to allow for more users, you would need to set up more servers, configure & tune, shard, test and then do it all again. This is costly and takes time. Sometimes, when applications suddenly become a hit, they have to scale quickly and can't afford downtime.
Data is auto spread across enough hardware to deliver single digit millisecond latency so databases are no longer the bottleneck. In the past, database layers couldn't keep up as you scale, so you would have to build bigger and bigger database clusters. This would slow down your application and become, frankly, a big web by mess. Now you can replicate data across availability nodes so it is available even if something goes down.
Take Pinterest, for example. They used Amazon web services to help them autoscale and they are now at 18 million unique visitors as of the end of February 2012 with only a dozen staff and one CTO. Without this type of service, he would have never been able to scale their systems as fast as they did. Arguably, this may have been detrimental to what is now one of the largest social networks of 2012.
Technically, Pinterest contains an entire users account information in a single shard instead of spreading it over multiple locations which a lot of other systems often do. Using the single shard method, when a database server is too full, they can move the shards to as many physical servers they need to without a problem. They are now running on 64 servers in multiple zones around the world.
Transformation four: The cloud puts a super computer in the hands of every developer
Supercomputers today are the privileges of the elite. They are expensive, if you get to use them your time is rationed and its often only available for the highest value jobs. By using the cloud, Amazon web services built the 42nd fastest super computer in the world and made it more available and more affordable for everyone.
The business example presented by Cyclecloud in conjunction to this transformation, led to applause from the audience and rightfully so. The company built a 50,000 core supercomputer with Amazon web services and worked with Schrödinger, a company that offers molecular-modeling and drug-design software, to run a virtual screen to identify proteins that are responsible for cancer.
Normally a super computer infrastructure costs at least 20 million so this is out of the reach for most researchers. By using cloud services with 3000 servers at full capacity, Schrodinger were able to do 112 years worth of calculations in three hours at the cost of $4828.85 an hour. This means any academic institution can now do bigger science faster and really make the impossible big science possible.
Transformation five: Experiment often & fail quickly
Yes, we've heard this one before. Facebook's mantra has been spread around and quoted in too many presentations to count. Many corporations and businesses need the lean approach. This way they can get the product in the hands of customers early on and leverage the tremendous power of the consumer. Though 95% of startups don't make it but it's still worth doing.
Transformation six: Big data without big servers
Traditionally, in order to store massive data volumes you'd need a huge data warehouse and invest in expensive server clusters to process the data. In order to make these work though, you need to know what it's being set up for.
Now, you often may not know the questions you want to ask. But with the cloud, you just have to load your data, organize and analyze the data and then visualize the results.
Etsy, Playfish, Foursquare and Yelp all have analytics at their core but have to also be flexible in how they analyze their data which is one of the reasons they use cloud services.
Transformation seven: A mobile ecosystem is needed for a mobile first world
This transformation is one that is more future facing. Building mobile is hard. An application may need to involve rich media, multi-devices, be location and contextually aware, real time presence driven, social graph based, rely on user generated content and recommendations, need a built in virtual goods economy, be integrated with social networks, or contain advertisements.
With a cloud mobile ecosystem, companies can use tools like Panda, Social Gold, Simplegeo, Directed Edge, Echo, Animoto, Twilio and others. This makes application building easier and they are available through Amazon's new marketplace which it launched yesterday to help companies find, buy and run software running on Amazon web services.
For Dr Werner Vogels entire keynote with case studies from PBS, Pinterest, Washington Post and Cyclecloud, you can now watch it in its entirety on Youtube.