Rules Are Not Controls

Many repos now have a rules file: CLAUDE.md, AGENTS.md, a constitution, a house standards doc. A team writes down how work gets done, the agent reads it on every turn, and the standards get followed. That is the outcome people expect.

A rules file is a prior. It shifts probability. It binds nothing. It is easy to treat that probability shift as a guarantee.

HANDBOOK.md Measures the Gap

A benchmark called HANDBOOK.md landed on July 28.¹ Sixty-five tasks, each one putting an agent inside a simulated company with real tooling: email, chat, calendar. Each task hands the agent a binding policy document, 20 to 124 pages, and asks it to do routine work in finance, medical billing, insurance, logistics, HR. Grading runs on 824 programmatic criteria across the set.

The best configuration scored 36.2 percent under strict grading, which requires every criterion to be satisfied. Most frontier models came in under 25.

The setup does not use jailbreaks, prompt injection, or tricks. The handbook is in the context window, and three out of four times the agent completes the job while breaking the rules it was given.

Two failure modes dominate. Agents lost rule detail across long sequences, and agents prioritized in-context requests over standing policy.

The Standing Order Loses to the Live One

That second one is the architecture working as designed.

Your policy and the user’s message arrive the same way. Both are text in a window. Nothing in the architecture stamps one as law and the other as a request. The policy is older, further back, more general, written for nobody in particular. The message is newer, closer, specific, and addressed to right now. The architecture favors recent and specific information over old and general information, which lets a model respond to you. A responsive agent and one that treats page 40 of a handbook as inviolable are coupled properties of the same architecture.

A policy line competes for attention with everything else in the window on roughly even terms. Adding policy does not automatically improve compliance: more pages can dilute the pages already there. The model’s attention is a real budget, not a courtesy it extends to you.

Teams often respond to a violated rule by writing it harder: bold it, capitalize it, move it to the top, add IMPORTANT and NEVER. Emphasis does work inside a turn, on a point the model can act on. It does not convert a preference into a constraint. It may raise the odds on one line while lowering them elsewhere in the file.

A stronger model changes the number, not the category. A model that scores 60 instead of 36 still has a failure rate. When a failure is costly, a lower error rate is not equivalent to a control.

The Same Mistake One Layer Down

On June 30, researchers disclosed GuardFall: a bypass that defeated the command-safety checks in ten of the eleven open source coding agents they tested, tools carrying about 548,000 GitHub stars between them.² Continue was the only agent that mitigated it.

The bypass relies on interpretation order. The agent checks the command string against its blocklist, decides it is safe, and hands the string to bash. Bash then rewrites the string before running it. A filter looking for rm never sees r''m, because the empty quotes only disappear at the moment of execution. No CVE was issued. The researchers classed it as a dangerous convention rather than a bug because it relies on an unsafe pattern.

The guard sits at the boundary. It returns yes or no, it logs, and it can be audited. But it inspects a description of an action rather than the action itself, and the shell interprets that description. Like a rules file, it asks the system to evaluate an action through its own representation of it.

What a Control Looks Like

A control makes the bad outcome unavailable. The test is whether the action remains possible when the instruction is ignored.

A read-only credential. The delete cannot be issued, so the delete does not happen. A container with no route out. The exfiltration has nowhere to go. Branch protection with a required check. The merge does not land. A token scoped to one repository. The other repositories are not in the world the agent can see.

These controls do not need the model to cooperate. They remove a capability rather than build a smarter guard. We automate what we can verify, and the corollary is that we can only delegate what we can bound.

Compare “never commit secrets,” “always run the tests before pushing,” and “do not modify files outside src/” with that list. All three are good instructions. Each has a failure rate that remains unknown until it is measured or an incident reveals it.

The File Still Helps

A prior that moves an outcome from 60 percent to 90 is valuable, and a rules file is a low-cost way to make that shift. It also gives an agent context the code cannot: which of two competing patterns won the argument, what the build needs, and why an obvious approach fails here. That is context engineering. I lean on it heavily.

The risk comes when writing a rule down is treated as resolving it. The rule can then perform at an unknown rate for months. An untested policy leaves its actual compliance rate unknown.

Two Piles

Rules-file lines fall into two piles.

Preferences. Style, naming, structure, tone, defaults, which library we use. Rules files are useful here. Eighty or ninety percent compliance is fine here. A miss costs a review comment.

Constraints. Anything where one violation is expensive: secrets, production, money, deletions, customer data, anything that sends mail to a human.

Each constraint needs a mechanism that lives somewhere other than the file. The line stays written down as well, because the prior is nearly free and it does help. But the line is not the control. Without an external mechanism, the constraint still depends on the model following the instruction.

Panavas et al., HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following, July 28, 2026. Sixty-five tasks across five professional domains, 824 programmatic criteria, handbooks of 20 to 124 pages. ↩
GuardFall, disclosed June 30, 2026. Affected tools included Aider, Cline, Goose, Open Interpreter, OpenHands, opencode, Plandex, Roo-Code and SWE-agent. Continue was the only one tested that mitigated it. ↩