LILY Margin Notes, part 2: Essence and accident

The technology stack that helps developers create new software has been evolving since the inception of computer science. In 1986, Fred Brooks, who had led IBM's System/360 operating system project and written The Mythical Man-Month, split the difficulty of building software into two parts: the problem itself and the work the tools add. He bet the tools had little left to take, so no single new tool could make building software ten times easier. He was right about writing programs. The tools that have been running them, have been adding work ever since.

An essay from LILY Labs

In 1986, Fred Brooks made a bet that the software industry has spent forty years trying to lose.

Brooks had run the operating system project for IBM's System/360 in the 1960s, written The Mythical Man-Month about what went wrong, and founded the computer science department at the University of North Carolina. He knew what it cost to build software. At the IFIP congress in Dublin that year he presented a paper called No Silver Bullet, reprinted in IEEE Computer in April 1987, and in it he wagered that no single development, in technology or in management, would deliver even a tenfold improvement in productivity, reliability or simplicity within a decade.

The decade passed without a tenfold gain. Thirty years later, nobody has managed to conquer that claim.

Brooks take is usually read as pessimism, when, in reality, it is quite the opposite. He was not saying software is hopelessly hard. He was saying that most of what used to make it hard had already been removed, and that what remained was the problem itself. In the first part of this series, The technologies that disappear, we asked which parts of infrastructure are allowed to disappear. Brooks drew the line that answers it, forty years before anyone needed it for this.

The problem and the tools

Brooks borrowed his distinction from Aristotle. For Aristotle, every entity has an essence, the properties it cannot lose without ceasing to be itself, and accidents, everything else that happens to be true of it. A man may be pale or tanned and remain the same man. He cannot stop being an animal and remain a man.

Brooks turned this into a test for software, and it still works. Ask of any piece of work: if it changed, would the program be a different program? If yes, it is the essence: the data, the relationships, the algorithms. If no, it is the accident: the language you wrote it in, the machine you fitted it to, the wait for a compile. The essence is what you are solving. The accident is what you solve it with.

The essence, he argued, has four properties that nothing can remove. It is complex, with more states than any machine. It must conform to interfaces other people designed. It is under constant pressure to change. And it is invisible: there is no drawing of a program that shows it whole.

Every big gain in the history of programming, he then pointed out, had come from attacking the accident. High-level languages freed programmers from registers, bits and disks. Time-sharing freed them from waiting a day for a result. Integrated environments freed them from gluing tools together. It is the same list of jobs that stopped being jobs that we told in Set and forget, seen from inside the decade it happened.

Then came the arithmetic. Even if nine tenths of the required effort were caused by accidents, and you removed every one of them, you would gain at most tenfold. And Brooks did not believe the share was anywhere near that high any more. The easy accidents had already been removed. What was left was mostly essence, and essence does not yield to a tool.

What outlasts the bet is the line it draws. An accident is whatever would vanish if the tools were perfect, and the rest of this essay applies that test to running software.

He was right about the essence

Brooks named the candidates of his day and discounted each: object-oriented programming, expert systems, program verification, graphical languages, better workstations. Every one of them attacked accident and so could only return what accident had left. None delivered tenfold in a decade. The essence held.

It still holds. Deciding what the software should do, getting it to conform to the systems and regulations around it, changing it without breaking it: that is where the hard part sits, and it sits there for a one-person company exactly as it did for IBM. Set and forget ended on the same point from the other direction. Deciding what to build is still the hard part.

The newest candidate is code written by AI. Brooks's test applies to it as to the others. It attacks the accident of writing, which is the part he said was already thin. That is a real gain and not a small one. It is not a gain on the essence, and anyone who has watched a generated feature pass every test and still be the wrong feature knows where the line is.

The accident grew back

Brooks was measuring the accident of writing a program. In 1986, running one meant handing it to the machine. There was nothing between the two to measure.

There is now. Open the repository of any production service and look at the files that are not the program. The Dockerfile. The Helm chart. The ingress rules, the IAM policies, the autoscaling thresholds, the readiness probe, the pipeline definition, the Terraform modules describing a cluster the application does not know it lives in. Not one of them describes the problem the application itself is trying to solve. Every one describes the conditions under which the solution is permitted to exist. By Brooks's definition that is accident, pure and entire: difficulty that attends production and is not inherent in the thing produced.

And it has grown the four properties of essence without being essence. It is complex: a cluster has more states than the application on it. It demands conformity: every file obeys an interface somebody else designed and will redesign. It changes constantly, which is the limit we found in part 1, where a platform that moves every week can never be learned into invisibility. And it is invisible: nobody can draw the running system whole.

This is the answer to the question part 1 left open. The machine still asks for attention because the accident is where attention goes. In Set and forget we cited a vendor's finding that operations teams spend most of their time troubleshooting. Brooks lets us say what kind of time that is. It is accidental time, spent on the conditions of the solution rather than the solution.

Where operating is the essence

The line does not fall in the same place for everyone, and the counterargument deserves its full weight.

For some systems, running is the product. A database company sells durability under failure. A content delivery network sells placement and latency. A payments processor sells the guarantee that a transfer happens once. For them, failure handling, replication and placement are not conditions of the solution. They are the solution, and no platform should take that work away from the people who understand it.

The threshold is the customer. Where the running of the system is what the customer pays for, operations is essence. Where the customer pays for what the system does, operations is accident. Most applications sit on the second side of that line. They use a database; they do not sell one.

Brooks adds a second warning that cuts against us. Abstractions that hide complexity, he wrote, often hide essence with it. A platform that removes the accident of running software must not remove the ability to reason about how it fails. What is at stake when a request is lost is a question about the problem, and it stays with the person who owns the problem. Nothing in this essay argues otherwise.

What is left when the accident goes

Brooks closed his paper with prescriptions for the essence, and they read as current forty years later. Buy software rather than build it where you can. Grow a system incrementally instead of building it in one piece. Find great designers and give them room. Everything he recommended is work on the problem. Nothing he recommended is work on the conditions.

Our own work at LILY sits on the accident side of Brooks's line. The aim is that the files describing where software runs become unnecessary, so a repository is left with the files describing what it does, and the questions a deployment asks are questions about the application. That is as much as is worth saying publicly, and it does not touch the essence. Deciding what the software should do, and conforming it to the world it serves, stays exactly as hard as Brooks said it was.

So the accident can go. But a system still has work that belongs to it: retries, checks, the handling of what fails. Where should that work live, in the middle of the system or at its ends? Jerome Saltzer, David Reed and David Clark answered that in 1984, and the next part reads them.

Brooks bet against a silver bullet for building software, and won. The bet he never had to place was the one about running it. That accident is still on the table.

You build. We carry.

Start for free. Read the docs. Talk to us about migrating a workload.

Key sources

• Frederick P. Brooks Jr., University of North Carolina, No Silver Bullet: Essence and Accidents of Software Engineering, Information Processing 86, IFIP, 1986; reprinted in IEEE Computer 20(4), April 1987
• Frederick P. Brooks Jr., No Silver Bullet Refired, in The Mythical Man-Month, anniversary edition, Addison-Wesley, 1995
• LILY Labs, The technologies that disappear, October 2026, and Set and forget, October 2026