Shipping Assumptions: A Reliability Stack for AI Code
Exploring the reliability crisis in AI-assisted development where clean code hides invisible assumptions, and solutions using C4, TLA+, and DST.
Stock photo for illustration only, not from the actual event
- AI-assisted software development accelerates code generation, creating gaps between components.
- Generated code that passes basic linting and tests can still hide implicit assumptions and boundaries.
- Managing reliability requires a structured stack including C4 Model, TLA+, and DST.
- Humans must retain the role of verifying intent and building shared models with automation tools.
The code looked good. It linted cleanly. The shallow tests passed. Everyone felt that we had built the right thing. Then the edge case appeared in production.
The developer treated it as normal operating procedure: bugs happen, tickets arrive, patches ship. QA was asked why they had not caught it. But QA never received a model of the system—only an implementation full of assumptions they were expected to reverse-engineer.
This is becoming the defining failure mode of AI-assisted development. We can generate code faster than we can understand the systems it creates. The danger is no longer confined to a bad function or an obvious syntax error. It lives in the space between components: boundaries, state transitions, failure modes, and invariants.
We are not merely shipping code. We are shipping assumptions we can no longer see. A linter can tell us whether code follows a set of local rules. A test can tell us whether selected examples produce the expected result. Neither can tell us what the system must preserve unless someone states it first.
Stock photo for illustration only, not from the actual event
Relying on generative AI for coding reduces upfront costs and development time significantly, but it introduces structural risks. AI often produces locally plausible results while lacking a holistic system architecture. Adopting formal modeling tools acts as an anchor connecting human intent to machine output.
An invariant is one of those statements: a condition that must remain true across every valid state of the system. An account balance cannot be changed without a corresponding transaction. A private object cannot become public without authorization. Two successful writes cannot silently erase one another.
When invariants remain implicit, clean code can still assemble into an incoherent system. Each component may look reasonable in isolation while the failure waits in a transition, a retry, a race, or a boundary nobody thought to draw.
That is the space AI-assisted development is rapidly filling with code. There was no golden age in which every developer wrote a formal specification before touching an editor. Most teams did not. Earlier software was full of hidden assumptions too, and nostalgia has a habit of removing the inconvenient parts of the past.
But the field did develop disciplines for reasoning above the code: architectural models, state machines, formal specifications, model checking, fault injection, and deterministic simulation. We tended to reserve them for systems where failure was obviously expensive. Everywhere else, rigor was treated as a cost to minimize.
"C4 maps the system. TLA+ states what must remain true. DST tries to make it false."
Dev.to
The C4 model gives software teams a hierarchy for visualizing a system: context, containers, components, and code. It works like a map with zoom levels. At the widest view, we see the system, its users, and neighboring systems. Zooming in reveals applications, data stores, services, components, and relationships.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment