My background in emergency services

I had been building ISPs and internet infrastructure since I was in high school. I was fortunate to be part of an early success, and after my first IPO in 1999 I stepped away from tech because I wanted to serve people directly. I enrolled in fire academy and trained as a firefighter and EMT. The fire service is a special perspective I have brought to everything I have built since.

In 2001 I joined Amazon as a systems engineer and ended up with the title Master of Disaster, responsible for the availability of every property bearing the Amazon brand. I trained software developers using fire-department incident management techniques and built a program of full-scale exercises called GameDay, where we deliberately took data centers offline so teams could practice handling failure under pressure. GameDay was the most visible piece of that body of work. From it I built three connected practices: modern Incident Management, and what we now call Site Reliability Engineering and Chaos Engineering. Netflix later adapted GameDay and named their version Chaos Engineering.

In September 2005 I took unpaid leave from Amazon, organized a 26-person volunteer task force, and deployed to Hancock County, Mississippi after Hurricane Katrina as part of the FEMA deployment. We were tasked with deploying emergency shelters that served as transitional housing for families, medical centers, and points of distribution.

I returned with many lessons that applied both to my work in technology and in emergency mangement. As I told BusinessWeek in 2008:

“One of the interesting things with being a pretty senior technology person operating in a disaster is that you get to see the state of the art versus the state of the practice.”

One of the big lessons was in mapping technologies. My team argued with the Red Cross about Google Maps that showed the I-90 bridge still standing. The bridge had been gone since the storm. Mikel Maron took that gap to OpenStreetMap, where anyone could update the map in real time.

In 2012 I convened the first Web Ops / Fire Ops summit at Artur Bergman’s loft in San Francisco, bringing engineers running large websites into the same room with fire-service incident commanders. Rob Schnepp, Ron Vidal, and Chris Hawley were the fire-service side of that room. They went on to form Blackrock Partners, train thousands of responders inside large technology companies, and write Incident Management for Operations for O’Reilly in 2017. I wrote the foreword.

In 2013 I cofounded Orion Labs to build voice software for frontline teams, using lessons from the fire service about how people actually communicate under pressure. Orion was one of the first applications fully certified to launch on FirstNet, the nationwide public-safety broadband network.

From 2015 through 2020 I combined tech and emergency service again, volunteering with Rock Medicine as Medical Command at SF Pride. Rock Medicine is an all-volunteer EMS organization covering large events across the Bay Area. Medical Command is the Incident Command System role that runs dispatch from the command post, coordinating volunteer EMTs and paramedics across a parade route serving hundreds of thousands of people. Rock Medicine was an Orion Labs customer, so the push-to-talk software I had helped build was running on the radios I was working.

Further reading

• MIT Technology Review TR35 — MIT Technology Review, 2011. The four-minute version of this arc, in my own words.
• Making Maps Work When Disaster Strikes — BusinessWeek, 2008
• The Do-Good Imperative — BusinessWeek, 2008
• GameDay: Creating Resiliency Through Destruction — USENIX, 2011
• Changing Culture and Being a Force for Awesome — O’Reilly Velocity, 2012
• An Oral History of #HugOps — Protocol, 2021