Amazon’s Elastic Block Store—A Glimpse of Cloud Computing’s Future

8/22/08

Earlier this week, Amazon announced an important new feature in its cloud computing Web service offerings that should further cause those of us who work in technology to consider the way in which we build software systems. It’s another exciting little glimpse of the future, one that should give pause to anybody still thinking of Amazon as just a bookseller.

In case you’ve recently emerged from under a rock, cloud computing represents a shift in thinking built on four pillars: open source software, virtualization, cheap commodity hardware and, most importantly, an acceptance of computing capabilities that exist outside the corporate firewall. In recent years, Amazon has transformed itself from an online store to a broad e-commerce platform that includes fulfillment and payment services, all while introducing an advanced suite of Web services that allow other businesses to incrementally scale up their computing needs. They’re now the poster child for cloud computing.

Amazon’s latest announcement was the addition of the Elastic Block Store (EBS) to their Elastic Cloud virtualized Linux box service. In Linux-speak, a block device is usually used to mount a storage device, such as a networked file system or a local disk, to a local server. Previously, EC2 virtual servers or “instances” were stateless—meaning all data on the server was lost when it rebooted or crashed. In order to keep data persistent, one needed to copy it to a durable storage system such as Amazon’s S3 service, which allows for reliable long-term storage of data. The new EBS makes this much easier and allows EC2 instances to create snapshots of data that are kept on S3. If the EC2 instance reboots, your data is safely stored on S3 (at least from the point of the last snapshot). This greatly simplifies previous methods of dealing with data on EC2 and will make it more attractive to developers.

The notion of using network-attached storage devices on Linux machines is not new, but something here is. In the Olden Days (pre-2006), figuring out how to attach a terabyte of highly redundant storage to one of your machines usually required some wrangling. It was an expensive proposition that involved heady fixed costs that were usually hard for any entrepreneurial spirit to swallow. If you needed 100Gb today and would need 700Gb later in the year, you were stuck with the option of buying 1Tb of disk up front and using it gradually. Today, it’s a pay-as-you-go system that can scale up to use some serious storage in a manner that’s very easy to access from a traditional Linux filesystem.

It’s not all roses, however. EC2 instances sometimes suffer from erratic or slow disk I/O, although Amazon advertises that the EBS performs better than EC2 local disks. You currently cannot share an EBS between multiple EC2 instances. In other words, you can’t spin up twenty EC2 servers and have them mount a shared filesystem like you can with a traditional networked filesystem in your datacenter. This is a complex problem, but Amazon has the technical chops to work on it and I expect them to look closely at this for future EBS releases.

The question that system architects should be asking is “does my architecture allow me to take advantage of services like EBS?” My company, StyleFeeder, has tens of millions of images hosted on Amazon’s cloud computing services. We were able to migrate our data quite easily primarily because key parts of our infrastructure were designed with a level of abstraction that requires only minimally invasive software surgery. Once you start working with pay-as-you-go cloud computing services, it’s important to think creatively about how you can use them. Removing rigid structures in your systems isn’t a small task, to be sure, but working in this direction can help revolutionize what your business is capable of and how quickly you can move it forward.

Philip Jacob, a software architect and veteran of several Boston-area startups, is the founder of Stylefeeder. Follow @

By posting a comment, you agree to our terms and conditions.

  • Pingback: StyleFeeder Tech Blog » Blog Archive » Amazon’s new elastic block service

  • Pingback: Whirlycott / Philip Jacob » Amazon EBS

  • http://www.pervasivepersuasion.com WebDesignMiami

    Kudos to the Cloud Crowd for Re-Inventing the Wheel!

    One thing 30 years in the IT industry has taught me is that the more things
    change, the more they stay the same. Another is that the only memory we
    seem to access is short-term. A third is that techno-marketeers rely on
    that, so they can put labels like “revolutionary” and “innovative” on
    platforms, products and services that are mere re-inventions of the wheel
    … and often poor copies at that.

    A good example is all the latest buzz about “Cloud Computing” in general and
    “SaaS” (software as a service) in particular:

    http://tinyurl.com/6let8x

    Both terms are bogus. The only true cloud computing takes place in
    aircraft. What they’re actually referring to by “the cloud” is a
    large-scale and often remotely and/or centrally managed hardware platform.
    We have had those since the dawn of automated IT. IBM calls them
    “mainframes”:

    http://tinyurl.com/5kdhcb

    The only innovation offered by today’s cloud crowd is actually more of a
    speculation, i.e. that server farms can deliver the same solid performance
    as Big Iron. And even that’s not original. Anyone remember Datapoint’s
    ARCnet, or DEC’s VAXclusters? Whatever happened to those guys, anyway…?

    And as for SaaS, selling the sizzle while keeping the steak is a marketing
    ploy most rightfully accredited to society’s oldest profession. Its first
    application in IT was (and for many still is) known as the “service bureau”.
    And I don’t mean the contemporary service bureau (mis)conception labelled
    “Service 2.0″ by a Wikipedia contributor whose historical perspective is
    apparently constrained to four years:

    http://tinyurl.com/5fpb8e

    Instead, I mean the computer service bureau industry that spawned ADAPSO
    (the Association of Data Processing Service Organizations) in 1960, and
    whose chronology comprises a notable part of the IEEE’s “Annals of the
    History of Computing”:

    http://tinyurl.com/5lvjdl

    So … for any of you slide rule-toting, pocket-protected keypunch-card
    cowboys who may be just coming out of a fifty-year coma, let me give you a
    quick IT update:

    1. “Mainframe” is now “Cloud” (with concomitant ethereal substance).

    2. “Terminal” is now “Web Browser” (with much cooler games, and infinitely
    more distractions).

    3. “Service Bureau” is now “Saas” (but app upgrades are just as painful,
    and custom mods equally elusive).

    4. Most IT buzzwords boil down to techno-hyped BS (just as they always
    have).

    Bruce Arnold
    Web Design Miami Florida