How Adversarial Instructions Cross Agent Boundaries

We built a multi-agent pipeline that retrieved customer records, summarized them, and routed the results to a downstream reporting agent. The retrieval agent was sandboxed. The summarization agent had no external tool access. The reporting agent could only write to an internal dashboard. Each individually looked fine. Then we found that an attacker had injected a directive into a customer record field - a field our retrieval agent treated as data - and watched it propagate, unmodified, through the summarization step and execute as an instruction in the reporting agent. The attack crossed two boundaries we thought we'd secured. Neither boundary was the entry point. Neither was the execution point. The damage happened in the middle.

Cross-boundary adversarial instruction propagation is not a hypothetical. It's what happens when you build layered systems and assume security compounds. It doesn't. This piece walks through what we've learned: how boundaries fail, why trust assumptions are the real vulnerability, and what defensive patterns actually hold up under pressure.

Understanding Adversarial Instruction Propagation

An adversarial instruction in an agentic context is any directive designed to override intended behavior, bypass safety constraints, or manipulate how an agent uses its tools - regardless of where it originates or how it's packaged. The payload may arrive as user input, as data retrieved from an external source, or as the output of another agent. The form doesn't define the threat. The intent and the effect do.

Agent boundaries exist wherever information or instructions move between components: API calls between agents, shared memory or context stores, tool invocation chains, orchestration layers that coordinate agent behavior. Each of these is a potential crossing point. In a single-agent system, the attack surface is one model, one context window, one tool set. In a multi-agent system, the attack surface is every interface between agents, and the number of those interfaces grows faster than teams typically audit them.

Cross-boundary attacks are harder to detect than single-agent attacks because the attacker's payload doesn't have to be coherent at any single point. A directive can be fragmented across multiple messages, encoded in a format that looks like benign data at the boundary where it enters, and reassembled by the receiving agent's context processing before it executes. By the time it triggers, it may look nothing like what crossed the boundary. Standard input validation that checks for known injection signatures at the entry point will miss it entirely.

Why Adversarial Instructions Propagate Across Boundaries

The root vulnerabilities are architectural. First: most multi-agent systems don't validate instructions at internal boundaries. The implicit assumption is that upstream agents have already filtered malicious content, so downstream agents can accept their outputs without inspection. This assumption fails the moment any upstream agent processes attacker-controlled data - which, in most production systems, happens constantly. Second: context accumulation means that each agent in a chain may carry forward conversation history, retrieved data, or prior instructions. An adversarial payload embedded early in that context persists through every subsequent boundary, often without any agent recognizing it as out-of-place.

The trust assumption problem deserves direct attention. Multi-agent systems are usually built with an internal/external distinction: external input gets filtered, internal agent-to-agent communication is trusted. This made sense when agents were simple, deterministic components. It doesn't hold when agents process natural language, call external tools, retrieve data from user-controlled sources, or generate outputs that feed directly into other agents' instruction spaces. Internal communication is only as trustworthy as the most compromised node in the chain.

Instruction obfuscation is what makes this exploitable at scale. Adversarial instructions may be base64-encoded in a data field. They may be split across multiple retrieved documents that, individually, look innocuous. They may be embedded in metadata that an agent's tool returns but that wasn't authored by any attacker-facing input - just a downstream consequence of an earlier injection. We've seen payloads that took four agent hops to fully reconstruct. Each individual hop would have passed a naive content filter.

Validate Instructions at Every Boundary

The principle is simple: treat every agent-to-agent communication as a potential attack surface. An agent being internal does not make its outputs safe. An agent being trusted at system design time does not mean its outputs remain trustworthy at runtime.

Concrete implementation looks like this. Agent A calls Agent B with a function like retrieve_user_data(user_id, filters). Instead of Agent B accepting filters as-is from Agent A, Agent B validates the filters parameter against a schema before processing: expected types, permitted values, structural constraints. If the filters contain a string that matches patterns associated with instruction injection - even if Agent A passed them without modification - Agent B rejects the call and logs the failure. Agent B doesn't trust that Agent A checked. Agent B doesn't assume the schema was validated upstream. Agent B checks.

Yes, this adds latency. Validation at every boundary is not free. In practice, we prioritize: validate all calls to agents that access sensitive data, that invoke external tools, or that can write to persistent state. Lower-risk internal agents with narrow, read-only capabilities can operate with lighter-touch validation. The goal is not uniform overhead - it's proportional scrutiny at high-consequence boundaries.

Isolate Agent Context and Instruction Space

Each agent should operate with a minimal, explicit set of instructions and capabilities. When Agent A calls Agent B, the call should pass only the specific data needed for that operation - not the user's full request, not the system instructions Agent A received, not the conversation history that informed Agent A's decision to make the call.

In practice, context compartmentalization means designing your agent interfaces the way you'd design a well-scoped API: the caller passes what the callee needs, nothing more. If Agent B's job is to format a report, it needs the report data. It does not need to know that the user originally asked for a data export or that Agent A retrieved the data from a particular source. Every additional piece of context you pass is an additional surface for an embedded instruction to persist into Agent B's reasoning.

The tension is real: some agents genuinely need context to make good decisions. A routing agent that needs to decide which downstream agent to call may need to understand the user's intent. The way to thread this is explicit context schemas - define what context a given agent is permitted to receive, enforce that at the boundary, and log deviations. If a routing agent needs intent, pass a structured intent field with permitted values, not the raw user message. You give the agent what it needs to function; you don't give it an unrestricted view of the full session.

Monitor and Log Cross-Boundary Instruction Flow

Every agent-to-agent call should record: which agent made the call, which agent received it, the input parameters, the results of any boundary validation, and the output or action taken. That's the minimum. Without this, post-incident reconstruction is guesswork. We've spent hours tracing propagation chains through systems where the only available signal was the final effect - an unauthorized write, a data exfiltration, an unexpected tool invocation. With boundary logs, that work shrinks to minutes.

Real-time detection matters more than post-hoc forensics. The patterns worth alerting on: Agent A rejects an input as invalid, but a structurally similar input reaches Agent B via a different path and gets accepted. An agent calls a tool it has access to but has never called in production before. An agent's output contains instruction-like patterns - imperative constructions, references to ignoring prior context, explicit commands - even though its role is purely data transformation. None of these is definitive alone. Together, they're a propagation signature.

Log everything at high-risk boundaries. At lower-risk boundaries, log anomalies and validation failures. Storage overhead from comprehensive logging is a real operational cost; the mitigation is tiered retention, not reduced coverage at critical points. An agent that has write access to a production database should have every call logged with full parameters, indefinitely or until the system is retired. An agent that reformats text can have lighter retention. Tier the logging to the risk, not to convenience.

Implement Explicit Instruction Boundaries

An explicit instruction boundary is a mechanism that prevents instructions - directives, commands, prompts - from being interpreted as such when they cross from one agent's data space into another agent's instruction space. The simplest version: data that an agent receives from an external source, or from another agent, should be typed and processed as data, not as instructions. The LLM backing Agent B should not be in a position to treat a string retrieved by Agent A as a directive to itself, because the interface design should make that impossible.

Consider the attack scenario directly. An attacker injects user_input = 'DROP TABLE users; -- ignore previous instructions'. Without explicit boundaries, Agent B receives this as part of its context and may act on the embedded directive, depending on how its prompt is constructed. With explicit boundaries, the interface between Agent A and Agent B is typed: Agent B receives a structured object with defined fields, and the user input string is stored as a value in a data.user_input field that Agent B's prompt explicitly labels as untrusted user data, not as an instruction. The model's context is constructed so that the field is presented as a data artifact, not as a command it should follow.

The limitation is genuine: this works well for structured, typed interfaces. In systems where agent-to-agent communication is natural language - where Agent A produces a summary and Agent B receives that summary as part of its instruction context - the boundary is harder to enforce. There's no clean structural separation between "Agent A's output" and "Agent B's instructions" when both are prose. In those systems, we rely more heavily on monitoring and validation, and we're honest that the defense is weaker. We don't yet have a reliable mechanism for preventing instruction smuggling through natural-language agent outputs in all cases.

When to Seek Support or Escalate

Escalate when you detect a cross-boundary attack in progress or after the fact, even if the impact appears contained. The attack you can see is probably not the only attack. Escalate when you're building a multi-agent system and can't map the full boundary surface - that's not a gap you should accept as temporary. Escalate when you've implemented validation and monitoring but lack visibility into whether they're actually blocking propagation attempts, because absence of alerts is not the same as absence of attacks.

Before you escalate, prepare: the logs showing how the instruction propagated across which boundaries; a timeline from initial injection to final effect; a list of agents that processed the payload at any point; a description of what each agent did with it. This gives whoever you're escalating to - your security team, an incident response firm, a platform vendor - an actual incident to work from rather than a vague concern.

Escalating a security issue is not admitting failure. Agentic systems are new enough that most teams are defending against attack classes that have no established playbook. Adversarial instruction propagation is one of them. The right response to discovering a propagation attack is documentation, escalation, and architectural review - not quiet remediation that leaves the underlying boundary design intact. We've seen teams fix the specific payload that exploited a boundary without fixing the boundary itself. That's a mistake made easier when escalation feels like exposure.