Agentic AI Security: What Actually Breaks in Production

Your board approved the agent pilot, and the vendor deck had a slide titled “Enterprise-grade security” listing SOC 2, encryption at rest and a prompt-injection filter. Interest in agentic AI security has exploded this year, and almost everything written about it is a glossary entry published by a company that sells a security product. This post is what that slide leaves out.

Almost every serious agent failure of the past eighteen months happened in software that worked exactly as designed. A support assistant read a customer email, a browser agent read a web page, a coding agent read a bug report, and each did what the text said, using permissions somebody had granted on purpose.

A chatbot’s worst case is a bad sentence. An agent’s worst case is a bad action, taken with real credentials, at machine speed.

What follows is a dated table of what has actually broken, an honest account of which controls are oversold, one design rule you can apply without an engineer in the room, and the distinction most writing gets wrong: your agent being hijacked versus an attacker wielding agents against you.

What agentic AI security means (and why it is not LLM security)

Agentic AI security is the practice of limiting what an AI agent can do when, not if, it is manipulated. It differs from LLM security because an agent does not just produce text: it reads untrusted content mid-task, holds real credentials, and acts through tools in a loop.

LLM security asks whether a model produces harmful words. Agentic AI security asks what happens when the system around that model is handed a hostile instruction while holding your access tokens. Three properties create the gap.

• It reads untrusted content mid-task. A web page, a ticket, an email or a tool’s return value arrives in the same stream as your own instructions, and a language model cannot reliably tell the two apart. That is prompt injection: an instruction hidden in content the agent was only supposed to read.
• It holds credentials and tools. The agent is not describing an action to a person who then performs it. It calls the API itself, with a token that works.
• It loops. One bad step becomes the input to the next, so a single manipulation compounds across a run rather than ending with one wrong answer.

If that line is still blurry inside your own product, the breakdown of an AI agent versus a chatbot is where to start. A chatbot that hallucinates costs you credibility. A hijacked agent spends money, deletes records or emails your data somewhere.

The OWASP Top 10 for Agentic Applications, in business terms

OWASP, the open community behind the web application security list your engineers already use, published its Top 10 for Agentic Applications on 9 December 2025, with more than a hundred contributors. It is the closest thing to a vendor-neutral map of ai agent security risks, which matters because most other lists are shaped by what their author sells. OWASP’s own follow-up research found that prompt injection touches six of the ten categories.

Read it as a risk register, not a checklist. Where no public incident cleanly demonstrates a category, it says so.

AI agent security incidents, 2025–2026

Nothing below is a thought experiment. Every row was publicly reported and dated. Read the last column first: the lesson is almost never to buy a better filter. Two patterns carry the weight: hijack risks exist because instructions and data travel in one channel, and supply chain risks are new because your dependency tree now holds instructions written in English.

The last two rows belong to a different conversation, and conflating them with the rest is the main dishonesty in current writing. GTG-1002 and the Mexican government breach are attackers using agents as tooling: one operator running reconnaissance at a speed that used to need a team. They say the cost of attacking you has fallen. They say nothing about whether your own support agent will leak data.

The other eight rows belong in your design review. In each, the victim’s own agent was pointed at attacker-controlled content and obeyed, or a component it trusted was hostile from the start. When a vendor cites an AI-orchestrated espionage campaign to sell a runtime product, the evidence and the pitch describe different threats.

The one design rule to remember: the lethal trifecta and the Rule of Two

In June 2025, Simon Willison named the pattern the lethal trifecta: an agent is exposed when one session combines private data, untrusted content, and a way to communicate externally. Any two are survivable. All three, and a successful injection has everything it needs: something worth taking, a channel to be told to take it, and a route out.

In October 2025, Meta’s engineering team turned the same observation into a rule a non-engineer can enforce in a design review.

An agent session should have at most two of these three: untrusted input, access to sensitive data, and the ability to act or communicate externally. If it needs all three, it should not run unsupervised.

The rule asks about architecture, not about the model, so you can put it to any vendor in ninety seconds. Two examples show how it lands.

Worked example: an email triage agent

It reads a shared inbox, which is untrusted input from anyone on the internet. It looks up the sender in your CRM, which is sensitive data. It sends replies, which is external action. That is all three, and it is the shape of both EchoLeak and ForcedLeak. Remove one property and the design is defensible again: route drafts to a human queue instead of sending them.

Worked example: a coding agent with shell access

The repository and its credentials are sensitive data. Issues, dependencies and fetched documentation are untrusted input. A shell with network access is external action. The fix is a sandbox: no production credentials, no live database, and no outbound network beyond a pinned package source. The Cursor bypass is why the boundary must be the sandbox, not an approval list inside a machine that can already reach production.

Which agentic AI security controls actually work, and which are oversold

Zenthos builds agents for a living and sells no security product, so this can be said plainly once: what reduces risk here is design and fewer permissions, not a purchase.

• Use the Rule of Two as a gate, not a guideline. An agent that needs all three properties gets scoped down, or runs with a human in the loop.
• Give every agent short-lived credentials, scoped per tool. Scope access to the task, not to the person who launched it, and never to a staff member’s OAuth token.
• Require human approval for irreversible actions only: money, deletion, external publishing, permission changes. Confirming everything trains people to click yes without reading, which is worse than no gate.
• Design two lanes of context. The user’s instruction is privileged; anything fetched from a page, ticket or tool result is not, and lane two must never choose which tool runs next.
• Control egress with an allowlist you actually own. EchoLeak and ForcedLeak were both exfiltration through a trustworthy-looking URL, and ForcedLeak’s allowlist contained an expired domain an attacker bought.
• Sandbox the agent and separate development from production data. This turns a catastrophic deletion into an annoying one.
• Log at the level of actions, not conversations: who started the run, the instruction, each tool call and its arguments, the data touched, the outcome.
• Keep an inventory and a tested kill switch. You cannot revoke what nobody wrote down, and an incident is a poor time to learn the switch does not work.

Now the list a security vendor cannot publish.

• Prompt-injection classifiers sold as a solution. Anthropic reported that mitigations on its browser agent cut attack success from 23.6% to 11.2% in autonomous mode. A control that halves a risk is a layer, not an answer. An October 2025 paper using adaptive attacks broke all twelve jailbreak defences and all eight indirect-injection defences it tested, and OpenAI has said injection is unlikely to ever be fully solved.
• Allowlists treated as boundaries. The Cursor vulnerability of January 2026 inverted one: shell built-in commands ran without approval, poisoned the environment, and an allowlisted command then carried the payload. The list became the auto-approver.
• “The model is aligned” as a control. Alignment governs what the model wants to do, not what your infrastructure permits, and only the second is enforceable.
• Red-teaming once at launch. An agent’s tool surface changes every sprint, so a test from six months and four connectors ago is a document, not a defence.

Identity, logging and the attribution gap

Non-human identity means a credential belonging to software rather than a person: a service account, an API key, an agent’s own token. Most pilots skip it, because reusing the launching employee’s session is the fastest route to a working demo. The cost arrives later, in two places.

First, the agent inherits every permission that person holds, which is never the set it needs; CISA and its international partners called this privilege creep in their May 2026 guidance, alongside what they call obscure event records. Second, the audit trail now blames a human for an agent’s actions, so when an investigator asks who approved the refund, nobody knows.

The fix is unglamorous: give each agent an identity, scope it per tool, keep tokens short-lived, and log at the action level with enough context to reconstruct a run months later. The fields worth capturing are in the guide to AI agent audit trails and SOC 2, and if nobody owns agent observability, that is the gap our LLMOps practice fills.

MCP servers and the agent supply chain

MCP, the Model Context Protocol, is the standard way to plug a tool into an agent: a small server advertises capabilities such as send email or query CRM, and any compatible agent can call them. It is also a package manager for capabilities, with everything that implies.

The evidence arrived fast. In September 2025 a cloned MCP server on npm quietly copied every email it processed to an attacker. In July 2025 a coding assistant extension shipped a destructive prompt merged through a rogue pull request. In February 2026 hundreds of listings on a skill marketplace turned out to be infostealers. None needed a novel exploit; each needed somebody to install a tool without reading it.

Three habits handle most of this: pin versions rather than tracking the latest release, keep an internal registry of approved connectors so nobody installs from a search result, and diff every update, remembering that the dangerous change may be a sentence of English rather than a line of code. The detail sits in our notes on running MCP servers in production, and teams who would rather own their connectors use our MCP server development work to build them in-house.

What regulators already expect

There is no agent-specific rulebook yet, and waiting for one is the wrong plan. In May 2026, CISA and its Five Eyes partners published the first joint multi-government guidance on adopting agentic AI. It names four risks a board will recognise: expanded attack surface, privilege creep, behavioural misalignment and obscure event records. Its instruction is blunt: avoid broad or unrestricted access, and start with low-risk, non-sensitive use cases.

NIST has a project mapping its existing security controls onto AI systems, but the agent-specific overlays were still unpublished as of September 2026, so there is no NIST agent standard to comply with today. For the obligations that already apply to you by law, the EU AI Act checklist for AI agents turns the text into decisions.

Two analyst predictions belong in the budget conversation. Gartner predicted in June 2025 that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls; that figure came from a poll of roughly 3,400 webinar attendees, a self-selected audience rather than a random sample. Gartner predicted again in April 2026 that by 2028 a quarter of enterprise generative AI applications will see at least five minor security incidents a year, up from 9% in 2025. Both are projections. The signal is the failure mode: projects die of weak controls as often as weak models.

Eight questions to ask before an agent goes live

These work on a vendor and on your own team, and a product owner can ask every one. Listen for specificity.

• Which of the three trifecta properties does this agent have in one session? Good: it names all three and says which was removed. Evasive: “it’s sandboxed.”
• Whose credentials does it act under, and what can it touch? Good: a scoped service identity, short-lived tokens, an enumerated tool list. Evasive: “it uses the logged-in user’s permissions.”
• Which actions are irreversible, and which need human approval? Good: an explicit list somebody can defend. Evasive: “the user can always review the output.”
• Show me the log of one real run, end to end. Good: the actual trace, with prompt, tool calls, data touched and decision. Evasive: a dashboard screenshot.
• What untrusted content enters the context, and from where? Good: an inventory of emails, tickets, pages and tool returns. Evasive: “we validate inputs.”
• When prompt injection succeeds, not if, what is the blast radius? Good: a worst-case walkthrough with a bound on it. Evasive: “our guardrails block prompt injection,” which alone is disqualifying.
• Where do the tools come from, and what happens when one auto-updates? Good: pinned versions, an internal registry, a diff process. Evasive: “we use the official ones.”
• How do we turn it off, and how fast? Good: a named kill switch, a tested revocation path, an owner. Evasive: “we’d disable the integration.”

If it helps to walk in with these on paper, you can download the pre-launch agent security checklist, a one-page version with go/no-go items on the front and these patterns on the back.

What this changes about budget and timeline

The expensive mistake is treating security as a line item bolted on after the agent works. By then it assumes broad access, and narrowing it means rebuilding what your stakeholders have already seen demoed.

In practice, scoping tools, deciding which actions need approval, separating the two context lanes and defining the logging schema is one to two weeks of design at the front of a project. Set against a rebuild, or against an agent deleting production data during a code freeze, that is cheap, and it keeps a pilot out of the cancelled column. If you are sizing this, the ranges in our note on what AI agent development actually costs assume the design phase is included, which is how we scope agentic AI builds rather than treating hardening as a change request.

FAQ

Can prompt injection be fixed?

No, not fully. OpenAI has said publicly that prompt injection is unlikely to ever be completely solved, and an October 2025 academic paper broke every defence it tested using adaptive attacks. Treat injection as a permanent condition of the environment and design so that a successful injection cannot reach anything expensive.

Are AI guardrails enough to secure an AI agent?

No. Guardrails, meaning classifiers that try to spot hostile instructions before the model acts, reduce attack success rather than remove it. Anthropic measured browser-agent injection success falling from 23.6% to 11.2% with mitigations applied. That is a worthwhile layer, and a disqualifying answer if it is the only one on offer.

Do AI agents need their own identity?

Yes. An agent acting on an employee’s token inherits every permission that person holds, and the audit log then records the person rather than the agent. Give each agent a scoped, short-lived identity of its own so access can be limited, reviewed and revoked without touching anybody’s user account.

Is agentic AI security required for SOC 2 or the EU AI Act?

Neither names agents explicitly, but both apply. SOC 2 auditors expect access control, change management and logging that cover every identity, including non-human ones. The EU AI Act assigns obligations by risk level and by role, regardless of whether a person or an agent performs the task in question.

What is the single highest-value agentic AI security control?

Removing one of the three properties from a session: untrusted input, sensitive data access, or the ability to act externally. It costs nothing to license, cannot be defeated by a cleverer prompt, and it is the only control that still limits the damage after every other defence has failed.

Should we pause our AI agent rollout?

Usually not. Pausing forfeits ground and teaches you nothing. The better move is to narrow the scope: begin with low-risk, non-sensitive use cases, as CISA and its international partners recommended in May 2026, and add sensitive data or autonomy only once logging, approval gates and a kill switch work.

Designing agents that are hard to hijack

None of this argues against shipping agents. It argues for shipping ones whose worst day is survivable. The teams still running agents in two years will be those that decided early which of the three properties each session may hold, gave every agent its own identity, and logged enough to reconstruct a bad run.

Zenthos designs and builds agentic AI systems with the Rule of Two applied at the architecture stage rather than bolted on before launch, along with the identity, tooling and observability that make an agent auditable. If a pilot is waiting on your approval, get in touch for a free consultation and bring the vendor deck. We will tell you which of the three properties the design still needs to lose.