I Stopped Writing Rules for Coding Agents That CI Could Not Enforce
A developer shares why repository instruction files are not enough for coding agents and shifted toward executable gates enforced in CI.
Stock photo for illustration only, not from the actual event
- Repository instruction documents are often ignored without automated enforcement mechanisms.
- Effective automated gates should always originate from real failures that already happened.
- Explicit inline comments allow tracked exceptions without invisible bypasses.
- Agents Playbook features 13 zero-dependency reference gates covering common repository governance needs.
The author previously treated repository instructions in files like AGENTS.md as the finished product, explaining the architecture, banning certain patterns, requiring named exports, and instructing coding agents to keep files small. However, despite an excellent document, the very next pull request could still violate those exact rules.
This failure is not always the agent's fault, as a rule existing only as prose requires every contributor to remember it at precisely the right moment. Review becomes the first place the repository discovers the rule was ignored, prompting a shift toward treating instructions as explanations and gates as enforcement.
Stock photo for illustration only, not from the actual event
The tempting response of automating everything creates a wall of noisy checks that nobody understands and everyone learns to bypass. Instead, the new approach starts with a failure that has already happened, such as an agent using any to quickly push a TypeScript change through, violating the repository rule requiring unknown and runtime input validation.
Transitioning from prose rules to automated checks represents a crucial evolution in software engineering alongside AI coding assistants. Because human review bandwidth is limited, binding policies directly to CI pipelines eliminates cognitive fatigue and stops repetitive mistakes before they reach human reviewers.
The developer published the executable gates from Agents Playbook as a zero-dependency CLI. For instance, testing the no-any rule locally using:
npx --yes @agentskit/playbook@0.1.0 run no-any --cwd /tmp/playbook-gate-demo
allows the command to exit non-zero and point directly to the violation. Furthermore, the reference gate accepts explicit same-line exceptions such as // allow-any: upstream SDK has no types, making the decision visible and allowing repositories to maintain a baseline count of escape hatches.
Stock photo for illustration only, not from the actual event
The Playbook currently includes 13 zero-dependency reference gates covering concerns such as secrets, file size, named exports, ADR requirements, internationalization, design tokens, native HTML, and PR intent, keeping governance closely attached to verifiable evidence.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment