Architecture Realities: The Gap Between Architecture Diagrams and Reality

Every team has a diagram. It lives in a wiki page, or a slide from an architecture review eighteen months ago, and it shows the system as a set of labelled boxes connected by arrows. New joiners are shown it in their first week. It gets pasted into design docs. Somebody occasionally says “let me pull up the architecture” and shares their screen.

And almost everyone who has worked on the system for more than a few months knows, in a vague unexamined way, that it is not quite right.

The interesting question isn’t why diagrams go stale. It’s that even a perfectly maintained, up-to-the-minute diagram would still leave out most of what determines how your system behaves — because the gap isn’t mainly about accuracy. It’s about what the notation can express at all.

The Diagram Shows Structure. The System Is Behaviour.

Boxes and arrows capture what exists and what talks to what. That is genuinely useful and it is a small fraction of what matters.

A diagram cannot show you that the call from the API gateway to the auth service takes four milliseconds at the median and eleven seconds at p99.9. It cannot show that when the recommendation service slows down, the retry policy in the client turns a degradation into an outage by tripling the load on something already struggling. It cannot show which failures are isolated and which cascade, where backpressure exists, or what happens to in-flight requests during a deploy.

The arrow between two services is the same width whether it carries ten requests a day or a hundred thousand a second.

Those behavioural properties are what you care about at 3am, and essentially none of them are representable in the notation. The diagram describes a system at rest. Systems are only interesting in motion.

It Was True on the Day It Was Drawn

The diagram is a snapshot of an intention. Someone drew it to describe what was going to be built, and at that moment it was accurate, because nothing existed yet to contradict it.

Then the system got built, under a deadline. A direct database read was added because going through the service API was too slow and the sprint was ending. A service that was supposed to be decommissioned after the migration still serves four percent of traffic, because the last consumer was never tracked down. A queue was introduced between two components to absorb a load spike, and it stayed. A second instance of something was spun up for a specific customer and never folded back in.

None of these are failures of discipline. Each was a reasonable local decision. But every one of them widens the gap, and nothing in the process closes it, because updating the diagram is nobody’s job and no build fails when it drifts.

Documentation that nothing depends on will always lose to work that something depends on. That is not a culture problem to be fixed with reminders; it is what happens when a thing has no consumer who notices when it’s wrong.

The Load-Bearing Things Nobody Draws

Beyond drift, there’s a whole category of components that never appear on any diagram at any point, including the day it was drawn.

The cron job on a VM that somebody set up in 2021 and that reconciles two datasets nightly. The shared database that two supposedly independent services both write to, which means they are one service wearing two names. The manual step where someone runs a script at month end. The spreadsheet a team maintains that feeds a config. The DNS entry pointing somewhere nobody can explain.

These are load-bearing. Remove any of them and something breaks in a way that takes a day to diagnose. And they are absent from the picture not through oversight but because the picture was drawn to communicate a design, and the design doesn’t include them. They are what happened to the design.

A useful exercise: ask the longest-tenured engineer on the team what isn’t on the diagram. You will get a list, delivered immediately, with no need to think about it. Everyone knows. It has simply never been anyone’s task to write it down.

The Arrows Are Where You Actually Live

Diagrams put the design effort into boxes. Boxes get names, owners, team allocations, sometimes a colour scheme. Arrows get a label if you’re lucky.

But the arrows carry almost everything that causes incidents. Is this call synchronous or asynchronous? What is the timeout, and does the caller’s timeout exceed the callee’s? Is the retry policy exponential with jitter, or three immediate retries that turn a blip into a thundering herd? What’s the serialisation format, and what happens when one side adds a field? Is delivery at-least-once, and is the receiver idempotent? What does the caller do when this returns a 503 — fail, degrade, or serve stale?

A box is something you can test. An arrow is a contract between two teams that nobody owns.

That ownership vacuum is why integration failures dominate incident reviews. Each team tested their box thoroughly. The arrow was assumed by both and specified by neither.

If you only improve one thing about how your team draws systems, annotate the arrows. Timeout, retry behaviour, failure mode, sync or async. It makes the diagram uglier and considerably more honest.

Read the Gap Instead of Closing It

The instinct on discovering the divergence is to launch a documentation effort and bring the diagram back in line. That’s usually a waste. It will drift again within a quarter, for all the same reasons, and you will have spent two weeks producing something with no consumer.

Better to treat the gap as information. Where the diagram and the system disagree is a map of where the design met resistance — where a boundary turned out to be wrong, where a deadline forced a shortcut, where a decision got quietly reversed. That divergence tells you more about how your organisation actually makes technical decisions than any retrospective will.

Three things are worth the effort. Generate what can be generated — service meshes, distributed traces and dependency graphs produce a picture from what is actually running, and a machine-drawn graph is ugly and correct, which beats beautiful and wrong. Date every diagram and name its audience, because a diagram without those is read as universal truth when it was only ever a simplification for one purpose. And draw the failure paths, not just the happy one, since the happy path is the part everyone already understands.

The diagram isn’t supposed to be the system. It’s supposed to be a useful lie. The damage comes from forgetting which parts are the lie.