Assume the Injection Succeeds

You can't stop every prompt injection targeting your AI agents. But you can limit the damage before one ever causes harm.

Call her Dana. She is a composite of the product security leaders I have spoken with this year, but the ticket in her queue is familiar at nearly every software company right now: an agentic feature is six weeks from launch and waiting on her sign-off.

The feature is a support agent that can look up customer accounts, review order histories, issue refunds below a set threshold, and update shipping details. Engineering is proud of it. Tasks that once took a full day can now be resolved in seconds, and three more agents are already lined up behind it on the roadmap. At first glance, the ticket looks like every security review Dana has handled over the past decade.

It is not, and Dana is the first person in the company positioned to see why.

The playbook fails quietly

Every security review Dana conducts starts with the same core questions: What are the inputs, and how are they validated? What did the pentest find? Can the issue be patched? What is the blast radius?

Those questions earned their place through decades of shipped software. Now watch what happens when Dana applies them to the agent.

What are the inputs, and how are they validated? For any other product component, Dana would enumerate the inputs and constrain each one. But the agent accepts open-ended natural language from anyone on the internet, and that input also functions as the program. A language model processes instructions and data in the same token stream, with no built-in mechanism for reliably distinguishing between them. Forty years of "never mix code and data" collapse inside a single component. There is no parser to harden.

What did the pentest find? A traditional pentest samples fixed behavior, but an agent's behavior is probabilistic. A prompt that failed safely on Tuesday tells Dana little about the ten thousand ways it could be rephrased. In 2023, Carnegie Mellon researchers showed that a search algorithm could generate adversarial suffixes that bypassed safeguards in aligned models, including model families the researchers had never queried. No one had to craft those attacks by hand. A point-in-time test cannot clear a search space that continually regenerates itself.

Can we patch it? When a jailbreak technique drops, there's no CVE, no advisory, no Patch Tuesday. The model provider retrains on its timeline. The incident unfolds on yours.

What's the blast radius? It's the only question on the template that still holds up. Keep it in mind.

Dana is not uncovering an implementation bug. The answer to "can this component be manipulated?" is yes, by design. That is not merely one reviewer being cautious. It reflects the field's broader consensus. OWASP now maintains a Top 10 for agentic applications, with prompt injection serving as the enabling weakness behind much of the list. That leaves Dana with two bad options. She can block the launch and become the reason the roadmap slips, or she can approve it and take responsibility for whatever a manipulated agent does with production credentials.

What the market gave her

When Dana looks for help, the products she finds all share the same basic architecture: guardrails, injection classifiers, and moderation layers that monitor the conversation and flag messages that appear malicious before the agent can act on them. Each makes a statistical judgment over open-ended language. As a first line of defense, that is sound, and the technology is improving every quarter.

I spent years at companies built to defend consumer platforms against automated attacks: Impermium against spam and abuse, Shape Security against credential stuffing and fraud. Both roles taught the same lesson. An adversary who can probe a statistical defense at will eventually finds the input that slips through. And that's not hypothetical for agents. EchoLeak, disclosed in 2025, was a zero-click prompt injection against Microsoft 365 Copilot. Instructions hidden in an ordinary email were crafted to evade the injection classifier, and succeeded. The agent's legitimate permissions handled the rest. One of the most heavily defended agents in production, and a single detection failure translated directly into action.

This reveals the real shape of Dana's problem. Strip away the novelty, and the agent is a familiar figure from security literature: the confused deputy, a privileged intermediary that can be tricked into using its authority on behalf of the wrong principal. Every prompt-injected agent is a confused deputy with production credentials. The frameworks these agents are built on repeat the same old mistake in a modern form. They treat giving the model access to a tool as permission to use that tool however it chooses. But capability wiring is not authorization.

Security has solved the confused deputy problem before, not by teaching the deputy to recognize con artists, but by limiting what the deputy is allowed to do.

The reframe

So Dana stops reviewing the model and starts reviewing the deputy. It is a shift the security industry has made before: assume breach, one layer up. In this case, assume the prompt injection succeeds.

The old review question, "Can this agent be manipulated?" already has an answer. Yes, by construction. The more useful question is, "What can a manipulated agent actually do?" That is a question Dana's discipline already knows how to answer. It is a question of blast radius, least privilege, and the parts of her existing review process that still hold.

Enumerate the authority. What can the agent access, and what does the feature actually require? The difference is unearned privilege. That difference becomes the attacker's budget.

Derive the boundary from intent. This support agent may issue refunds below $200, but only for valid orders. It may never change payout destinations. It may access records for one authenticated customer per session, and nothing in the conversation can push it beyond that boundary.

The same principle applies whether an agent is reading a single patient's chart or a single customer's transaction history.

Attack the boundary before launch. This is not a one-time red-team exercise. It requires an adversarial corpus that repeatedly tries to push the agent beyond its limits in a sandbox. Those tests should run again after every prompt revision and every new tool integration, because the boundary can shift whenever the agent changes.

Enforce at the call, deny by default. Every tool invocation should be checked against the boundary at the moment of action, before anything executes. The result should be the same whether or not an upstream system flagged the conversation. Simon Willison's "lethal trifecta" describes the dangerous combination of private data, untrusted content, and an outbound channel. When a feature cannot eliminate one of those three entirely, the tool-call boundary is where that combination must be broken, one action at a time.

None of this makes the model trustworthy. That is the point. It makes the model's authority finite, which is something Dana can test, enforce, and prove.

The review she can sign

Now look at the review artifact itself, the document that carries Dana's name.

Before, it said: "The vendor's guardrails are enabled, and we monitor for anomalous behavior." That is hope with a dashboard.

After, it says: "Here is the agent's intended behavior. Here is the operating boundary derived from that intent. Here are the adversarial campaigns we ran before launch, the boundary crossings we found, and the constraints that closed them. Here is the runtime enforcement point that holds regardless of whether an upstream system detects the attack. And here is the decision log that proves it."

That is a risk-acceptance memo built on evidence rather than adjectives. It is the same standard Dana would demand for any other component in the release.

The product launched on schedule. Security was not the reason it slipped. Security was the reason it shipped on time. And when the next three agents reached her queue, the review workflow was already in place.

That is the quiet win in this story. Dana's discipline was never obsolete. It was missing one essential thing: a boundary that exists as an enforced fact, not as a paragraph in a system prompt.

Where TenetGraph fits

None of Dana's playbook requires a vendor. It does require standing infrastructure that teams can maintain and rerun whenever an agent changes, especially when three more agents are already waiting in the queue.

TenetGraph turns that playbook into infrastructure. It derives each agent's operating boundary from the artifacts that already define it, including the code, system prompt, tool schemas, and granted scopes. Teams do not have to write the boundary by hand or reconstruct it after every revision.

Before launch, an adversarial engine tests the boundary. It runs again whenever the agent changes. At runtime, TenetGraph checks every action against the relevant constraints before execution and logs each approval, denial, and escalation with the exact constraint that produced the decision.

The security review gets evidence it can stand behind. The roadmap keeps its launch date.

Define the boundary. Authorize the action.

The bottom line

You will not make the model trustworthy. That is not a failure of the review process. It is the nature of the component.

What you can do is make the agent's authority finite, tested, and provable. That is, almost boringly, a product security problem. It is the kind of problem this discipline has solved before.

Detection determines how often the boundary is challenged. The boundary determines the cost when detection fails.

The teams shipping agentic features fastest are not necessarily the ones with the best classifiers. They are the ones that can already show, with evidence, exactly what their agents are allowed to do when every upstream control fails.

Chris Finan is co-founder and CEO of TenetGraph. He previously served as director for cybersecurity legislation and policy on the White House National Security Council staff, was a product director at DARPA, and held executive roles at Shape Security (acquired by F5) and Impermium (acquired by Google), defending consumer platforms against automated fraud and abuse.