Puppet, Chef Ease Transition to Cloud Computing
September 2, 2011
Organizations as diverse as Northrop Grumman (NOC), Harvard University, Zynga, and the New York Stock Exchange (NYX) have filled job websites with requests for talented puppeteers and master chefs. A quick dig into the job listings reveals that these positions have nothing to do with office entertainment or gourmet meals. Instead, the companies want people who have mastered Puppet or Chef, competing software tools that sit at the heart of the cloud computing revolution…
In essence, Puppet and Chef are levers used to control data center computers in a more automated fashion. The software has helped companies tap vast stores of computing power in new ways, accelerating research in fields such as financial modeling and genetics. “This really changes the way science gets done,” says Jason Stowe, the chief executive officer of Cycle Computing, a startup that uses Chef to configure thousands of computers at a time so that clients can perform calculations at supercomputer speeds. Before adopting Chef, doing such configurations took hours or even days. “We’re down to single-digit minutes now,” Stowe says.
The need for such tools originated with Google (GOOG), Amazon.com (AMZN), and their peers, who have long had to deal with the burden of managing tens or even hundreds of thousands of servers to support vast Web operations. Over the years these companies developed custom tools that can quickly turn, say, a thousand new servers into machines capable of displaying Web pages or handling sales. These programs allow the companies to run enormous, $500 million computing centers with about three dozen people at each one.
As more and more businesses move their software applications to the cloud, a handful of startups have developed mainstream versions on such data-center software. Puppet and Chef are the two with the highest profile. “The custom tools built by Google, Amazon, and some other guys were such closely guarded secrets,” says Jesse Robbins, co-founder of Opscode, the 20-person, Seattle-area startup behind Chef. The company has raised $13.5 million in venture capital. “Our founding thesis was to open up these tools to everyone else.”
Opscode’s Chef and its competitor, built by Puppet Labs, are both open source: Anyone is free to use and adapt the software. The companies make money by selling polished versions of the core technology and additional features, and by charging for advice on how to implement and best use it.
Luke Kanies came up with the idea for Puppet in 2003 after getting fed up with existing server-management software in his career as a systems administrator. In 2005 he quit his job at BladeLogic, a maker of data-center management software, and spent the next 10 months writing code to automate the dozens of steps required to set up a server with the right software, storage space, and network configurations. The result: scores of templates for different kinds of servers, which let systems administrators become, in Kanies’s metaphor, puppet masters, pulling on strings to give computers particular personalities and behaviors. He formed Puppet Labs to begin consulting for some of the thousands of companies using the software—the list includes Google, Zynga, and Twitter—and earlier this year he released the first commercial version. Kanies expects it to account for 50 percent of fourth-quarter revenues, and total 2011 revenues to be double last year’s. He’s raised more than $7 million from venture capitalists including Kleiner Perkins Caufield & Byers.
The big benefits of Puppet and Chef alike are time- and cost-savings. Stanford University used to rely on a hodgepodge of tools to manage its hundreds of servers, says Digant Kasundra, an infrastructure systems software developer at the university. They’ve since replaced that unorganized toolbox with Puppet. “We were a ragtag team, and now we are a cohesive unit, and our servers require a lot less attention,” Kasundra says. The number of employees needed to operate the machines drops as well—a critical advantage in an uncertain economy when companies are trying to keep a lid on payroll costs. Palo Alto-based Jive Software has used Puppet to double the number of servers a single engineer can handle. “It’s a huge impact for us,” says Matt Tucker, chief technology officer and co-founder of the company.
Rivalry between Chef and Puppet is fierce. Puppet Labs argues that its software requires less training and collects more data about what’s happening on the network. Chef claims a bigger developer base. What’s certain is that both efforts are attracting attention from industry heavyweights. Michael Dell, the founder and CEO of Dell (DELL), follows Kanies on Twitter. Traditional data-center heavyweights such as Hewlett-Packard (HPQ) and IBM (IBM) have shown interest in this type of software and could emerge as potential acquirers, says Mary Johnston Turner, an analyst at consultant IDC. Kanies, for one, says he’s happy being independent: “We are not going to be a scalp on someone else’s belt.”
The bottom line: Investors have bet $20.5 million that Puppet and Chef, competing server-management tools, will be at the forefront of cloud computing.