Agent Anomaly Detection Watches AI Agents at Work

Somebody Finally Built the Boring Layer

Google announced Agent Anomaly Detection on September 16, 2026, now in private preview on the Gemini Enterprise Agent Platform. It reads the traces that deployed agents already emit and decides whether an agent is operating outside its intended boundaries.

That is deliberately unexciting, and that is the point. Most of the security effort around AI agents so far has gone into the moment of the call — validating a prompt, checking a tool argument, gating a permission. What has been missing is the layer every other production system takes for granted: something watching what actually happened, after the fact, across sessions.

• Availability: private preview on the Gemini Enterprise Agent Platform, requiring Agent Development Kit 1.2 or later (2.1.0 or newer recommended)
• Coverage: four OWASP Top 10 for Agentic Applications risks — tool misuse, identity and privilege abuse, cascading failures and rogue agents — plus operational risks like resource exhaustion
• Architecture: three tiers — a lightweight statistical scan, an LLM reasoning layer, and an invocation-level analysis
• Output: findings with severity, a plain-language explanation and recommended action, published to Security Command Center and exposed via API

How Does the Three-Layer Design Keep Costs Sane?

This is the part worth studying if you are building anything similar. Running an LLM over every agent trace would be ruinous, so Google tiers it.

A cheap statistical pass runs across all sessions and flags the unusual ones. Only those sessions go to an LLM-based reasoning layer that examines what the agent was actually trying to do. If that layer needs more detail, a third layer drops to individual tool invocations. The expensive reasoning only fires where the cheap scan already found something odd — a straightforward funnel, and the same shape that makes log analytics affordable in conventional security operations.

The other design decision is that detection runs asynchronously, outside the live execution path. Nothing sits between the agent and its tools adding latency. The trade is honest: you find out after, not during. For a monitoring and audit layer that is the correct trade, because an inline check that slows every call down gets disabled the first time somebody benchmarks it.

What Kinds of Failures Does It Actually Catch?

The detector list maps onto the OWASP Top 10 for Agentic Applications (2026), which is a useful sign that this is built against a shared vocabulary rather than a vendor-invented taxonomy.

Tool misuse (ASI02) covers unsafe tool chaining, parameter manipulation and prompt injection reaching a tool call. Identity and privilege abuse (ASI03) covers an agent acting with more authority than its task warranted. Rogue agents (ASI10) covers behaviour that has drifted off its brief entirely.

Cascading failures (ASI08) is the one that will earn its keep soonest, because it is the least exotic. Infinite loops, oscillating retries, feedback amplification between agents — these are not attacks, they are ordinary bugs that happen to burn money and quota at machine speed. The operational risk detectors sit right next to them: resource exhaustion and escalating token usage. Most teams will meet those failures long before they meet an adversary.

Findings carry a severity, a plain-language explanation of what triggered them, and a recommended action. They land in Security Command Center for the security team and are also available through an API, so developers can programmatically halt or block agent actions above a severity and probability threshold. Google engineer Achuth Narayan Rajagopal says the team is working on letting customers define what counts as anomalous for their own business, which is the obvious next requirement — a normal trace at one company is a red flag at another.

Where This Sits in the Agent Security Stack

The useful way to think about this is defence in depth for agents. Identity and access control at the front — the approach behind Postman Passport's keyless API access and Zanzibar-style authorization — decides what an agent is allowed to touch. Runtime governance decides what leaves. Behavioural detection like this decides whether what happened made sense.

None of the three replaces the others, and the third has been the thinnest. Agents that plan, call tools and recurse produce failure modes that no static permission model anticipates, and one-click provisioning on an existing trace stream is a low enough barrier that teams might actually turn it on. More in our AI security coverage.

Sources: Google Developers Blog — September 16, 2026; Help Net Security — September 17, 2026.