AI Agent Guardrails Versus Controls: What Each One Can Actually Stop

Ask a team how its agent is secured and the usual reply is a list of filters. A classifier on the input, a classifier on the output, a long system prompt full of rules. Then ask what the agent's API key can delete. The pause that follows is the whole problem.

The error is not laziness. It comes from treating anything that looks like a gate as a gate. This page separates two things that share a vocabulary and do opposite jobs, then shows where each belongs when you design an agent. It stays on defence and names no attack recipes.

Why a filter feels like a wall

Define the terms first. A guardrail is any protection that works by shaping or judging what a model says or intends: a prompt rule, an input screen, an output classifier, a second model that reviews the first model's plan. A control is a protection enforced outside the model, which applies the same way whatever the model outputs: a narrow credential, a sandbox, an egress rule, a spend cap, an approval gate on a named action.

Guardrails are cheap to add, easy to demo and genuinely useful, which is how they earn trust they have not earned. Older security habits feed the confusion too: stand a screen next to the risky thing and move on. That works when the filter parses a fixed grammar. It works poorly when the input is natural language and the filter is itself a statistical model with an error rate.

Where the guardrail-only design fails

The clearest public evidence is a paper titled The Attacker Moves Second, from 2025, whose authors include Milad Nasr and Nicholas Carlini. Its complaint concerns how defences get evaluated: against a fixed set of known attack strings, or against weak optimisation that never saw the defence's design. The authors instead run adaptive attacks, with search, gradient methods, reinforcement learning and human exploration, and report that they bypassed twelve published defences, most at success rates above 90%, even though many of those defences had claimed near-zero.

Read that carefully. It does not show that every product fails. It shows that a vendor's headline number and the behaviour under a patient, adaptive opponent can sit far apart, and that the number is only as honest as the test that produced it.

Simon Willison's lethal trifecta frames the consequence. When an agent combines private data, untrusted content and a way to send data out, a probabilistic screen alone stands in the way of a leak, and you place that bet on every request. For a deeper look at why screening input alone falls short, read why input filtering does not stop prompt injection.

A cleaner split: odds versus consequences

• Guardrails shift probability. A bad attempt becomes rarer and you gain signals. System prompt rules, classifiers, content filters and refusal training all sit here.
• Controls shift impact. Damage stays capped if the model errs or is steered. Scoped credentials, read-only mounts, egress allowlists, sandboxes, mandatory approvals, rate limits and audit logs sit here.

The design rule that follows: every action with real impact has at least one control in front of it. A guardrail may stand beside the control. It may not stand alone.

Building it in order

1. Inventory the tools. Enumerate each callable tool and grade the harm it could do: private reads, writes, outbound sends, spending, deletion. Mark each protection as guardrail or control. Any dangerous tool whose only labels say guardrail is the first repair.
2. Cut one leg of the trifecta. Decide per agent whether you can drop private data access, untrusted input or outbound channels. Our production prompt injection defences article extends this.
3. Issue short-lived, task-sized credentials. Keep the secret out of the model's reach and let the tool layer enforce the scope.
4. Place approvals on actions. A reviewer must see the true recipient and amount. An approval that fires on everything trains people to click through.
5. Isolate execution. Limit filesystem, network and credentials for code and tool calls. The controls catalogue by layer shows where each of these lives.
6. Log both layers. Frequent guardrail triggers reveal where attempts cluster. A control that denies shows a guardrail that missed.

What design-level research adds

Two papers move from "filter better" to "constrain the architecture". A group of researchers from several institutions published Design Patterns for Securing LLM Agents against Prompt Injections in June 2025. It proposes patterns that limit what an agent can do once it has touched untrusted content, and it discusses what each pattern costs in utility.

The CaMeL paper, Defeating Prompt Injections by Design, goes further. It builds a protective layer around the model that separates control flow from untrusted data and uses capabilities to block unauthorised data flows. In the AgentDojo benchmark it solved 77% of tasks with provable security, against 84% for an undefended system, as the authors report. That seven-point gap is the price: a control costs capability, and the paper says so openly.

Do not count the reviewing model

Many teams add a second model that approves the first model's plan, and then score the design as controlled. That is still a guardrail. It is a probabilistic judge reading text the attacker may have shaped, and it fails under the same adaptive pressure the Attacker Moves Second authors describe.

The unwelcome part is the bill. Real controls remove capability, as the CaMeL gap above shows: the agent cannot do some of what it could before. Product owners push back, and they push hardest on the controls that matter most. Teams then keep the guardrail, restore the capability and call the result secure. My advice is to accept the lost capability on high-impact actions and keep guardrails for the low-impact rest.

A guardrail can stand alone in the right conditions. If the action is cheap to undo and the credential is already scoped so that the worst case is small, a guardrail alone is an acceptable risk decision, because a control would add cost without reducing a real loss. Judge each action by what a wrong call costs, not by principle.

How to test a guardrail honestly

Replaying yesterday's attack strings only tests your memory. Hire a tester who adapts to your actual filter, give them real time, and write down whatever gets through. The output tells you which actions still need a control behind the guardrail. It is not a score to advertise.

No layout removes risk entirely. Controls get misconfigured, and approvers get tired. The goal is a model mistake whose price is capped and whose trace is visible. Where an agent handles payments, production changes or regulated records, bring in a qualified security reviewer for the control layer before launch. Treat this page as guidance and not as an audit.

Questions about AI agent guardrails