How to Bid on Computing Power

October 13, 2011 Off By David
Grazed from MIT Technology Review.  Author: Cindy Walker.

Imagine buying time on a computer in Ireland or Indiana the same way you’d bid for an antique on eBay. That’s how a new crop of startup companies called "cloud brokerages" plan to change the way companies buy and sell computing capacity…

Cloud computing has already made accessing computer power more efficient. Instead of buying computers, companies can now run websites or software by leasing time at data centers run by vendors like Amazon or Microsoft. The idea behind cloud brokerages is to take the efficiency of cloud computing a step further by creating a global marketplace where computing capacity can be bought and sold at auction.

Such markets offer steeply discounted rates, and they may also offer financial benefits to companies running cloud data centers, some of which are flush with excess capacity. "The more utilized you are as a [cloud services] provider … the faster return on investment you’ll realize on your hardware," says Reuven Cohen, founder of Enomaly, a Toronto-based firm that last February launched SpotCloud, cloud computing’s first online spot market.

On SpotCloud, computing power can be bought and sold like coffee, soybeans, or any other commodity. But it’s caveat emptor for buyers, since unlike purchasing computer time with Microsoft, buying on SpotCloud doesn’t offer many contractual guarantees. There is no assurance computers won’t suffer an outage, and sellers can even opt to conceal their identity in a blind auction, so buyers don’t always know whether they’re purchasing capacity from an established vendor or a fly-by-night startup.

"It’s appealing for a lot of folks to get [cloud capacity] for a penny on the dollar, get it immediately, and turn it off just as quickly," says Jeffrey Kaplan, managing director at the consultancy ThinkStrategies. "But the scariness thereafter is the burden it places on the customer to ensure its success, security, and manageability."

Currently, Cohen says, 1,300 companies have registered to sell computing power on SpotCloud (another 2,100 people have registered as buyers). At any given time those sellers are offering the computing-power equivalent of 100,000 servers with 400,000 gigabytes of computer memory. To put that in perspective, AT&T owns only slightly more than 20,000 servers.

Sellers currently offering computer capacity on SpotCloud include Domicilium, a Web hosting company that built a 20,000-square-foot data center on the Isle of Man, a tax haven off the Irish coast. Cohen says he recently got a call from a data center that streams weekend games for a major sports league in the United States. The problem: "Most of the time the provider’s servers sit idle," he says. "We’re talking tens of thousands of servers that do nothing between Monday and Friday."

Exchanges such as SpotCloud aren’t yet attracting huge e-commerce companies looking to run critical software or websites. Instead, buyers are on the lower end of the market—companies looking for overseas data centers to test location-specific applications, or to run so-called batch computing operations on the cheap. According to Cohen, daily trading volume amounts to "several hundred" gigabytes of computer memory.

Another challenge facing exchanges is that different cloud services purchased on an exchange won’t necessarily work together. Kaplan says trying to build a computing environment from a hodgepodge of remote computers presents a "challenge [to] manageability" that would require businesses to invest heavily in software to monitor and manage those resources.

Several companies are now seeking to build exchanges that would both allow bidding and guarantee interoperability of computers from different vendors. ComputeNext, a Seattle-based startup, says it is developing software that will let cloud clusters communicate. Similarly, Germany’s ScaleUp Technologies is working on software called "Federated Cloud" that would let users sell capacity from different data centers in different regions through a single interface.