BusinessWeek by Olga Kharif and Ashlee Vance · · Article
"The custom tools built by Google, Amazon, and some other guys were such closely guarded secrets. Our founding thesis was to open up these tools to everyone else."
BusinessWeek's 2011 cloud infrastructure feature placed Opscode and Chef at the center of the enterprise cloud transition, covering how open-source tools were bringing operational practices from Google and Amazon to every company.
In September 2011, BusinessWeek reporters Olga Kharif and Ashlee Vance covered the infrastructure automation movement, featuring Jesse Robbins and Opscode alongside Puppet Labs as the two leading open-source tools transforming enterprise cloud adoption. The article captured infrastructure automation at the moment it crossed from the secret playbooks of Google and Amazon into the mainstream enterprise market — and established Robbins’ founding thesis in the business press.
Opening the Black Box
Google and Amazon had built some of the most sophisticated server automation tools in existence — and kept them entirely proprietary. The competitive advantage was too significant to share. When Jesse Robbins cofounded Opscode in 2008, his starting point was simple: what if everyone could have what only the hyperscalers had?
“The custom tools built by Google, Amazon, and some other guys were such closely guarded secrets. Our founding thesis was to open up these tools to everyone else.”
Chef translated that institutional knowledge into open-source software. Any company — a 20-person startup, a university, a Fortune 500 — could now manage thousands of servers with the same programmatic discipline that Amazon used to run its global infrastructure.
Infrastructure as Code, at Supercomputer Scale
The article’s most striking example came from Cycle Computing, a startup using Chef to configure thousands of cloud machines on demand. Where that process once took hours or days, CEO Jason Stowe told BusinessWeek: “We’re down to single-digit minutes now.”
The supercomputing application wasn’t hypothetical. Scientists could bring up a 10,000-processor cluster on Amazon’s cloud in 45 minutes, complete complex protein-binding research in eight hours, then shut the whole operation down — at a fraction of the cost of traditional supercomputer access. Infrastructure as code had made high-performance computing a pay-as-you-go service.
From Niche to Enterprise Standard
By September 2011, demand for Chef and Puppet expertise had spread to organizations as different as Northrop Grumman, Harvard University, the New York Stock Exchange, Stanford, and Zynga. Job listings for skilled “puppeteers” and “master chefs” were multiplying across industry.
Stanford replaced a “hodgepodge of tools” with Puppet, reducing the attention its servers needed. Jive Software used Puppet to double the number of servers a single engineer could manage. The operational leverage — fewer people running more infrastructure, with greater reliability — demonstrated why infrastructure automation was becoming essential.
The Competitive Landscape Takes Shape
Puppet Labs and Opscode were fierce rivals, but their competition validated the category. Michael Dell followed Luke Kanies on Twitter. HP and IBM — the traditional data-center heavyweights — were watching closely, described in the article as potential acquirers.
By the time BusinessWeek ran the piece, investors had put $20.5 million into Chef and Puppet combined — infrastructure automation had moved from underground practice to investable market.
Full Transcript
AI-generated
Data Centers, September 01, 2011 — Puppet, Chef Ease Transition to Cloud Computing
By Olga Kharif and Ashlee Vance
Organizations as diverse as Northrop Grumman, Harvard University, Zynga, and the New York Stock Exchange have filled job websites with requests for talented puppeteers and master chefs. A quick dig into the job listings reveals that these positions have nothing to do with office entertainment or gourmet meals. Instead, the companies want people who have mastered Puppet or Chef, competing software tools that sit at the heart of the cloud computing revolution.
In essence, Puppet and Chef are levers used to control data center computers in a more automated fashion. The software has helped companies tap vast stores of computing power in new ways, accelerating research in fields such as financial modeling and genetics.
"This really changes the way science gets done," says Jason Stowe, the chief executive officer of Cycle Computing, a startup that uses Chef to configure thousands of computers at a time so that clients can perform calculations at supercomputer speeds. Before adopting Chef, doing such configurations took hours or even days. "We're down to single-digit minutes now," Stowe says.
The need for such tools originated with Google, Amazon.com, and their peers, who have long had to deal with the burden of managing tens or even hundreds of thousands of servers to support vast Web operations. Over the years these companies developed custom tools that can quickly turn, say, a thousand new servers into machines capable of displaying Web pages or handling sales. These programs allow the companies to run enormous, $500 million computing centers with about three dozen people at each one.
As more and more businesses move their software applications to the cloud, a handful of startups have developed mainstream versions on such data-center software. Puppet and Chef are the two with the highest profile.
"The custom tools built by Google, Amazon, and some other guys were such closely guarded secrets," says Jesse Robbins, co-founder of Opscode, the 20-person, Seattle-area startup behind Chef. The company has raised $13.5 million in venture capital. "Our founding thesis was to open up these tools to everyone else."
Opscode's Chef and its competitor, built by Puppet Labs, are both open source: Anyone is free to use and adapt the software. The companies make money by selling polished versions of the core technology and additional features, and by charging for advice on how to implement and best use it.
Luke Kanies came up with the idea for Puppet in 2003 after getting fed up with existing server-management software in his career as a systems administrator. In 2005 he quit his job at BladeLogic and spent the next 10 months writing code to automate the dozens of steps required to set up a server with the right software, storage space, and network configurations. He formed Puppet Labs to begin consulting for some of the thousands of companies using the software — the list includes Google, Zynga, and Twitter — and earlier this year he released the first commercial version.
Stanford University used to rely on a hodgepodge of tools to manage its hundreds of servers. They've since replaced that unorganized toolbox with Puppet. "We were a ragtag team, and now we are a cohesive unit, and our servers require a lot less attention," says Digant Kasundra, an infrastructure systems software developer at the university.
Palo Alto-based Jive Software has used Puppet to double the number of servers a single engineer can handle. "It's a huge impact for us," says Matt Tucker, chief technology officer and co-founder of the company.
Rivalry between Chef and Puppet is fierce. Puppet Labs argues that its software requires less training and collects more data about what's happening on the network. Chef claims a bigger developer base.
Michael Dell, the founder and CEO of Dell, follows Kanies on Twitter. Traditional data-center heavyweights such as Hewlett-Packard and IBM have shown interest in this type of software and could emerge as potential acquirers.
The bottom line: Investors have bet $20.5 million that Puppet and Chef, competing server-management tools, will be at the forefront of cloud computing.