Resilience Engineering: Why Incident Reviews Become Compliance Theatre

Nobody sets out to build a process that wastes everyone’s time. Incident reviews start as the opposite — a team genuinely wants to understand what went wrong, and sitting down together afterwards is obviously the right way to do it.

Then somewhere between the tenth review and the hundredth, the thing quietly inverts. The meeting still happens. The document still gets written. The action items still get filed. And if you asked anyone in the room whether they learned something, you would get a pause before the answer.

The failure is structural rather than cultural, which is why it happens to teams who care. Each step that degrades the review is individually reasonable. Together they produce a ritual that generates artifacts instead of understanding.

Action Items Became the Product

A review with no action items looks like a review that failed. So every review produces some.

This sounds harmless and is the single most corrosive thing in the process. Once action items are the expected output, they get manufactured — “add more logging,” “update the runbook,” “investigate alerting” — and because they exist, they get tracked. Completion rate becomes the metric, because it is the only thing in the whole exercise that can be counted.

Within a year you have a backlog of several hundred low-value tickets nobody intends to close, which becomes its own compliance problem requiring its own cleanup initiative.

Meanwhile the most valuable thing a review can produce — four engineers now share an accurate mental model of a system they previously each understood differently — generates no ticket, appears in no dashboard, and cannot be reported upward.

A postmortem can be completed perfectly without anyone having learned anything.

Blameless Became Unsayable

Blameless postmortems were a genuine advance. The idea was that people will only tell you what actually happened if telling you carries no career risk.

What survived in most organisations is the vocabulary, not the safety. “Blameless” turned into a rule about grammar. You cannot write that an engineer deployed without checking the staging results, so you write that a deployment occurred. The passive voice spreads until the document describes a sequence of events with no people in it at all.

And the people are the data. The question that explains an incident is never what did they do — it is why did that look like the right thing to do at the time? They had reasons. The dashboard said something reassuring. The runbook described a different situation. They had done this nineteen times before without incident. Six other things were on fire.

Strip the human reasoning out and you are left with a chronology of events, which tells you what happened and nothing about why. You can read it twice and still not know how to prevent a recurrence, because you never found out what the system looked like from inside the decision.

“Root Cause” Is a Field, Not a Thing

Most templates have a field labelled Root Cause, singular, and the field has to be filled.

Complex systems rarely fail that way. They fail when four or five conditions line up — a config that was slightly wrong for a month, a retry policy that amplified instead of absorbing, a dependency that degraded rather than failing cleanly, an alert routed to someone on leave. Remove any one and nothing happens. None is individually the cause.

Forced to choose, people pick the last thing that changed, or the most fixable thing. Those are rarely the most explanatory. Then that one thing gets fixed, everyone marks the incident resolved, and the four conditions that actually produced the outage are still sitting there waiting for a different trigger.

The template did that. Not the team.

Every New Reader Removes a Sentence

The first reviews a team writes are written for the team. Then one gets read by a director. Then one goes to a customer. Then an auditor asks for the last twelve.

Each new audience changes what can be written, and the change is always subtractive. By the time a document has four audiences, it is written for the most dangerous one.

You cannot write “we knew this was fragile and chose not to fix it” in a document an auditor will read.

But that sentence is frequently the truest and most useful one available. The technical cause was a timeout; the actual cause was that someone raised this in planning eight months ago and it lost to a feature. A review that cannot say so will instead report the timeout, recommend tuning the timeout, and leave the prioritisation failure completely untouched — where it will produce the next three incidents.

The fix is not better writing. It is accepting that one document cannot serve learning and accountability at once, and writing two.

You Only Review the Expensive Ones

Most organisations trigger reviews at a severity threshold. SEV1 and SEV2 get one. Everything below passes without comment.

This is backwards from a learning standpoint. Severe incidents are rare, politically loaded, and attended by people who need a resolution rather than an explanation. Near-misses and minor degradations are frequent, low-stakes, and contain nearly all the same information about how your system actually behaves — at a fraction of the cost to examine.

The near-miss is the best teaching material you will ever get. Something went wrong, the system or a human caught it, and nobody is in trouble. That is the one condition under which people tell you everything.

Almost nobody reviews those.

What a Real One Looks Like

The test is simple and uncomfortable: did anyone’s model of the system change?

Not “did we find a cause,” not “did we file actions.” Did someone leave the room understanding something about how this thing behaves that they did not understand walking in? If not, you held a meeting.

Three things move a review back towards that. Put beliefs in the timeline, not just events — what people thought was happening at each point, including the parts where they were wrong, because the gap between belief and reality is the finding. Allow more than one cause, and let the list include organisational ones. And separate the document people need for accountability from the conversation people need for learning, because trying to get both from one artifact reliably produces neither.

None of that is expensive. It is mostly deleting fields from a template and asking a better question.

The reason it rarely happens is that compliance theatre is comfortable. It produces evidence of diligence on a predictable schedule, and nobody has to say out loud that the real cause was a decision somebody made for sensible reasons eight months ago.