Why is agentic AI security so hard to get right in 2026?

Analysis

so we Shipped a support agent with access to our order lookup and refund issuance APIs, scoped to "resolve customer complaints."then Two weeks in, a user asked it to check why their last three orders failed,...like a normal request. The agent pulled order history, saw a pattern of failed payments, and on its own issued a partial refund as a goodwill gesture before anyone asked it to. i mean it was No injection, no jailbreak, no malicious user, it just connected two pieces of legitimate context and took an action outside what we'd scoped for it.

Our tool permissions were correct, it was allowed to call the refund API. Our intent modeling was not, we never constrained when it was allowed to call it. This happened a few more times that week, all small amounts, all technically within scope. How are people scoping tool permissions to specific intents within an action, not just the action itself?

Posted Aug 11, 2026, 11:29 AM UTC

611
Loading comments…