Inside LLM Security Flaws: What the Latest Vulnerability Research Means for Agentic AI Systems

We built a customer-facing agent with access to a document retrieval tool and a CRM API. The system prompt defined its role clearly. We thought we had scoped its capabilities tightly. Then we ran a structured red-team against it and discovered that by embedding a specific instruction sequence inside an uploaded PDF, an attacker could redirect the agent's tool calls to endpoints we had never intended it to reach - and the agent would narrate what it found, in plain language, in the chat window. The vulnerability was not in the model. It was in the gap between what we thought we had constrained and what the model would actually do under adversarial input conditions.

This is the defining characteristic of LLM vulnerability research as it applies to agentic AI security: the findings rarely look the way you expected, and the blast radius is almost always larger than a single conversation turn. Recent work on large language model exploits has started mapping this territory with more precision. What follows is our read of what the research is showing, what it confirms about known failure modes, and what it means for teams building and operating agentic systems right now.

Understanding the LLM Attack Surface

The attack surface of a language model is not a single boundary you can draw around the inference endpoint. It is a collection of interfaces, each with its own failure characteristics: tokenization boundaries, the instruction hierarchy encoded in the context window, tool-calling interfaces, memory retrieval mechanisms, and multi-turn state management. Each of these is a potential entry point.

Tokenization boundaries matter because models process text as token sequences, not as semantic units. Adversarial inputs can exploit the gap between how text looks to a human reviewer and how it parses at the token level - a technique that shows up repeatedly in recent academic red-teaming work. The instruction hierarchy is a more fundamental problem: models are trained to follow instructions, but they have no cryptographic or structural way to verify which instructions are authoritative. The system prompt, the user message, and content retrieved from external sources all arrive in the same context window, and the model must infer their relative authority from position and framing.

Research from groups at major AI labs and universities over the past two years has confirmed what practitioners suspected: indirect prompt injection is a genuine and repeatable attack class. The finding that surprised us - and that the research confirms - is how reliably retrieval-augmented systems propagate injections. When a model retrieves content from an external source and incorporates it into reasoning, that content has the same structural position as user input. The model has no native mechanism to treat it with less trust.

There is a critical distinction that gets lost in vendor conversations: vulnerabilities in the model itself versus vulnerabilities in how the model is deployed in an agentic system. A jailbreak that succeeds against a bare model in a chat interface often fails when the same model is operating inside a constrained agent with a narrow tool set - but the converse is also true. Deployment adds new attack surfaces: tool interfaces, external data sources, inter-agent communication channels. A model that is relatively resistant to direct jailbreak may be straightforwardly exploitable through indirect injection once it has tools to call.

Why LLM Vulnerabilities Cascade in Agentic Systems

In a single-turn chat, the worst outcome of a prompt injection is a policy violation - the model says something it should not have said. In an agentic system, the same injection can trigger a sequence of tool calls, each expanding the attack's scope. The amplification mechanism is the agent's autonomy: its ability to take actions, call tools, modify its own memory or state, and pass outputs to other agents or systems.

We have traced chains that look like this: adversarial instruction embedded in user-uploaded content reaches the agent during a retrieval step, the agent follows the injected instruction and calls an API with attacker-controlled parameters, that API response is passed to a downstream agent that treats it as trusted context, the downstream agent acts on it. At no point does any single component flag the sequence as anomalous. Each step looks locally valid.

The trust boundary problem is the structural issue underneath all of this. In multi-agent architectures, Agent A's output becomes Agent B's input. Most systems we have reviewed treat that hand-off as implicitly trusted - Agent B accepts Agent A's output with the same level of trust it would give a system-level instruction. There is usually no validation step, no attestation mechanism, and no semantic check on whether Agent A's output is consistent with its stated purpose. The research literature is increasingly clear on this: multi-agent systems inherit and compound the injection vulnerabilities of their component models, and the compounding effect is often nonlinear.

Traditional input validation fails here for a specific reason: language models are stochastic and context-dependent. Blacklist-based filtering is brittle because semantic equivalence means the same attack instruction can be expressed in thousands of syntactically distinct ways. Filtering on known attack strings catches the attacks you have already seen. It does nothing for variants or novel formulations - and the research on LLM jailbreak techniques shows that novel formulations are consistently produced faster than defenses can enumerate them.

Strategy 1: Red-Team Your Agent Before Deployment

Structured red-teaming for agentic systems starts with enumeration: what tools does this agent have access to, and what is the worst-case outcome for each tool if the agent is instructed adversarially? This is the attack surface map. From there, you design adversarial inputs that target the instruction hierarchy directly - attempts to override or supersede the system prompt - and inputs that work indirectly, through data the agent retrieves or receives from other agents.

The checklist we use covers: direct jailbreak attempts against the system prompt; indirect injection through user-uploaded files, retrieved documents, and API responses; context confusion attacks that attempt to shift the model's understanding of its role mid-conversation; multi-turn escalation where each individual turn looks benign but the sequence builds toward a policy violation; and tool-calling abuse where adversarial inputs attempt to manipulate tool parameters rather than the agent's natural language output.

Instrumentation is not optional here. You cannot find vulnerabilities you cannot observe. Before red-teaming begins, log every prompt construction step, every tool call with full parameters, and every inter-agent message. The vulnerabilities consistently hide in the gaps between what the developer intended and what the agent actually does - and those gaps are only visible in the logs. We have run red-team sessions where the chat-level output looked completely normal and the tool call logs showed the agent attempting to reach external endpoints that were not in its declared tool set.

Strategy 2: Implement Robust Tool-Call Validation

Schema validation and semantic validation are not the same thing, and you need both. Schema validation asks: does this tool call match the declared signature? Does it have the right fields, the right types, the right format? This is necessary but not sufficient. Semantic validation asks: is this tool call reasonable given what this agent is supposed to be doing? A tool call that passes schema validation perfectly can still be an attacker-controlled API request with adversarial parameters.

The layered approach we settled on: strict schema enforcement with no tolerance for deviation (if the tool call does not match the declared signature exactly, it does not execute); rate limiting and quota enforcement per tool per agent per time window (this contains the blast radius if an agent is compromised - it limits how much damage can be done before detection); and a semantic plausibility check that compares the tool call against the agent's current stated goal. The semantic check is the hardest to implement and the most imperfect, but even a simple consistency check - does this tool call type make sense given the agent's current task? - catches a meaningful fraction of injection-driven misbehavior.

The false-positive problem is real and matters for adoption. Overly strict validation breaks legitimate agent behavior, and if the engineering team disables validation because it blocks too much, you have nothing. The way we tuned our rules: analyze a corpus of benign agent behavior first, establish a baseline of what normal tool-call patterns look like for each agent role, then add validation rules that catch deviations from that baseline without catching normal variation. This takes more time than setting arbitrary thresholds, but the rules that come out of it are actually respected in production.

Strategy 3: Isolate Agent Execution Contexts

Containment is the goal. If one agent in a multi-agent system is compromised, the compromise should stop there. That requires deliberate isolation: separate API keys, separate database connections, separate credential stores, and separate network contexts for each agent that handles sensitive operations.

The scenario that clarified this for us: Agent A (user-facing) receives user input and routes tasks to Agent B (data retrieval) and Agent C (analysis). Agent A is the highest-exposure component - it receives arbitrary user input, including potentially adversarial input. If Agent A shares credentials or API access with Agent B, and Agent A is successfully injected, the attacker now has whatever access Agent B has. If the agents are isolated, a compromised Agent A can only do what Agent A's own credentials allow - which should be narrow: route tasks, not retrieve data or perform analysis directly.

Practical isolation mechanisms: containerization, where each agent runs in its own container with its own filesystem and network namespace; credential scoping, where each agent holds only the credentials for its specific tool set and those credentials have the minimum permissions required; and message boundary enforcement, where inter-agent communication passes through a validation layer rather than directly between agent processes. Containerization is the most operationally expensive of these, but it is also the most complete. The credential scoping is often the easiest win and the most commonly skipped step.

Strategy 4: Monitor and Alert on Anomalous Agent Behavior

Defining anomalous behavior for agents requires a baseline. What does this agent normally do? How many tool calls does it make per conversation turn? Which APIs does it call, and in what sequences? How long does it take to respond? What is its normal error rate? Without baselines, anomaly detection produces too many alerts on too little signal, and the alerts stop being reviewed.

The anomalies that, in our experience, are most reliably indicative of compromise: a sudden spike in tool calls to sensitive APIs (the agent calling an authentication API it has never called before, or calling a data export function it has not used in any baseline session); repeated failed authentication attempts followed by a successful call from the same agent instance (a pattern that suggests the agent was instructed to try credential combinations); tool calls whose parameter content includes strings that were not present in any of the agent's inputs for that session (which suggests the agent is constructing parameters from injected instructions rather than from its task context); and unexpectedly large output sizes, which sometimes indicate data exfiltration attempts through the agent's natural language output channel.

The response workflow needs to be specified before an incident occurs, not during one. Anomaly detected means an alert reaches a human reviewer with enough context to make a decision: what was the agent doing, what specifically triggered the alert, and what is the agent's current state? The human needs to be able to throttle, isolate, or terminate the agent from that same interface. If the response requires three teams and two escalation chains, the agent will have completed its malicious task before anyone acts.

Strategy 5: Design Agents with Explicit Failure Modes

Agents should fail safely and visibly. The instinct in engineering is to build systems that fail gracefully and silently - that degrade without bothering the user. For agentic systems under adversarial conditions, this instinct is wrong. An agent that silently fails to execute an instruction it should have refused has obscured a security event. An agent that visibly refuses and logs the refusal has created an audit trail and a detection opportunity.

The scenario that makes this concrete: user attempts prompt injection - an instruction embedded in a document that tells the agent to ignore its system prompt and exfiltrate data. An agent designed with explicit failure modes recognizes the injection pattern (or fails to recognize a valid task context, which is a good proxy), refuses to execute, logs the attempt with the full context of what was received, and surfaces the event to whatever monitoring system is watching. The human reviewer sees the event. The attack fails and is documented.

Designing for explicit refusal requires three things: first, define the set of operations the agent is allowed to perform - this is the positive capability definition, and it should be narrow enough that the agent can actually evaluate whether a given instruction falls within it; second, define the set of operations the agent must refuse - the negative capability definition, which should include anything that would modify its own system prompt, exfiltrate data outside defined channels, or call tools not in its declared tool set; third, implement the refusal as a first-class behavior, not as an error handler. The agent should be able to produce a structured refusal response that includes what it was asked to do, why it refused, and where it logged the attempt.

Strategy 6: Establish Clear Governance for Agent Capabilities

The governance problem is a growth problem. Early in a deployment, an agent has a small tool set, access to limited data, and operates in a narrow domain. Over time, capabilities accumulate - new tools get added, new APIs get integrated, access scope expands to accommodate new use cases. Each addition is individually reasonable. Collectively, they produce an agent whose actual capability profile is substantially larger than anyone's current mental model of it.

A capability inventory is the starting point: a maintained record of what each agent can do, which tools it has access to, what credentials it holds, and what data it can read or write. This sounds obvious. In practice, we have reviewed production multi-agent deployments where no one could produce a current, accurate capability inventory for any single agent. The inventory should be versioned and change-controlled - adding a capability to an agent should require the same review process as changing an API integration.

Assign risk levels to capabilities based on worst-case outcomes: what is the worst thing that could happen if this capability is misused? A tool that reads public documentation is low risk. A tool that writes to a production database or calls an external API with customer data is high risk. Risk levels inform review thresholds: high-risk capability additions require security review before deployment; low-risk additions require documentation. The tension between capability and safety is real - more capability means more utility - but the goal is not to minimize capabilities, it is to make the trade-off explicit and reviewed rather than implicit and accumulated.

Strategy 7: Conduct Ongoing Adversarial Testing

Red-teaming is not a pre-deployment checkbox. Agents change - their tool sets expand, their system prompts get updated, their underlying models get upgraded. The LLM vulnerability research landscape also changes: new attack techniques are published, new jailbreak patterns are discovered, and your own understanding of your system's threat model improves as you operate it in production. A red-team that was thorough six months ago may miss attack classes that were not known then.

The testing cadence we recommend: a baseline red-team before initial deployment establishes a vulnerability inventory and a performance benchmark. Monthly