Magic, Silver Bullets & Value of True Engineering
Marc Andreessen from a16z wrote, or rather proclaimed, in 2011: “Software is eating the world.” The proclamation was fairly prescient for the world of consumer software at least. There are very few aspects of digital consumer’s life left untouched by the magic of software. You can almost feel it: when one tap on our smartphone can summon a car and driver; in software that recognizes our faces; when insomnia induced purchases show up at our doorstep next morning. Such seemingly magical examples abound.
What would be an appropriate reflection on “software eating the world” in 2020? The answer, I believe, is that we have never been more capable in expressing world’s problems in software.
But at the same time, we have had no material advances that have made it easier to build software. Anyone who has been in the trenches building software intuitively knows this. And, the great computer scientist Fred Brooks saw this problem even before the current software revolution had started. He described a timeless “essence” of building software, which he divided into “essential difficulties” and “accidental difficulties”.
Say, you want to build a mobile health app. To start, you need to understand the disease and its impact on patient’s life. You have to determine how to close the gap on existing care workflows, or if there is a need to redo some or all of them. You have to design a user experience, simple and compelling, to facilitate adoption of virtual care. All of this has to be represented as “highly precise” and “richly detailed” description for building the software system. The Essential Difficulties.
Then, there are other pieces to the puzzle. You have to make technology and architectural choices for the system. You have determine where to host, setup operations to build, manage and operate the software app. Last but probably the most important thing, you have to secure the human resources and organize teams to build, manage and operate the system. The Accidental Difficulties.
While we have become more capable with software, the essential never got, and by its very nature, never will get easier. And accidental, even though solvable, is simply out of control. I mean, there are multi-billion dollar companies just addressing subsets of those accidental tasks. In his 1986 paper, Brooks concluded he sees “no silver bullet” in the foreseeable future. Boy, was he right!
So if some of the consumer software examples seem magical, it is only because those companies empower teams to manage the essential tasks, and eliminate the accidental as much as feasible. And all that has to be done iteratively. Well engineered software solutions to complex problems, like creating health, are never “done”. You know the solution will change the environment, and may even cause new problems. The starting premise then is that solution will yield deeper insights about the problem, for which you will need to constantly improve upon.
There is simply no magic. To borrow from Robert Heinlein, “one man’s magic is another man’s engineering”.
Andreessen predicted healthcare and education sectors as the ones that will be eaten by software next. A decade hence, and with the benefit of the hindsight at our disposal, we can say that is far from happening. Software may eat the world but the world doesn’t just go away. The reality doesn’t lose the ability to surprise us. That is true wherever the purpose of the software has a critical interface with the human purpose. For the endeavor of human health, equally applicable to education, I have argued previously that software solutions for complex environments should enhance, not replace, human purpose. So how does one go about building software for complex environments?
- By framing building software as a systems engineering exercise, rather than an act of deploying technology to the current way of doing things.
- By understanding the design is not just a matter of creating UI/UX specifications, but about determining what aspects of software system need to be constantly adapted to the complexity of human environment.
- By prioritizing iterative framing of the problem and definition of a solution, over a fixed deadline-oriented software delivery of scope for an incomplete understanding of the problem.
- By acknowledging that software solutions for complex problems can only be evolved because there is neither a fixed problem, nor a final solution.
- By knowing that true value will come from solving the essential, for which software can only ever be means to an end.
As Brooks explained decades ago, there are no silver bullets for the above. But there are certainly silver linings. I will start exploring those next.
