Continuous production is an idea that’s probably as old as the first modern blast furnaces, an idea that decades of industrialization have perfected, an idea that has become all the more current in internet operations now that the mom-and-pop web hosting has turned into industrial scale operations. The body of literature regarding the topic of running an online service continuously without eating away profit margins is getting richer by the day, with a crucial contribution from James Hamilton (pdf of LISA paper) summarizing a wealth of traits of online operations that manage to scale from a technical and, as importantly from an economic standpoint.The peculiar nature of software is so that there are actually two distinct types of production:
- Production of service/software from need to formal requirements to design, development and release, the end production is a set of components/artifacts that are finished enough to fulfill users’ needs (a.k.a. software development)
- Production of service/software from these artifacts to an actual service (a.k.a. service production)
Software engineers and their managers typically care about the former while the latter has traditionally been the realm of system engineers and operations. These 2 crowds have quite different cultures, nigh antagonistic since developers are meant to create manageable change while operations is tasked with keeping the whole thing running around the clock. And any experienced person in operations will tell you that things break, in their vast majority, when they change, when they are brought out of steady state.
The aforementioned paper offers a way to bridge that gap by bringing developers much closer to operations. That’s Amazon’s supposed motto: “You build it, you run it”. This is a cultural change that has proven natural in a tiny structure where everyone does everything but is proving harder to keep as the company grows, especially when it has not been constantly pushed as the correct way of organizing work. It has been one of my ongoing projects to make sure that we close that precious feedback loop between developers and operations.
