Securing AI Agents That Access Sensitive Data

An internal support agent at a growing SaaS company had read access to the entire customer database — every account, every record, all of it — because that was the easy way to build it. One afternoon, a user asked it a cleverly worded question about "my account," referenced an order ID that wasn't theirs, and the agent cheerfully summarized a different customer's plan, billing history, and email address to a total stranger. Nobody hacked anything. The agent did exactly what its access allowed.

That's the AI agent security failure in a nutshell: the agent's permissions, not the attacker's skill, defined the blast radius. Let's tear down the weak design and rebuild it so a confused or manipulated agent simply can't reach data it shouldn't.

Before: The Weak Prompt

The original design gave the agent a database connection and trusted it to behave.

[object Object], ,[object Object],(,[object Object],):
    data = db.query(,[object Object],)      ,[object Object],
    ,[object Object], agent.run(
        ,[object Object],
        ,[object Object],
    )

What this does: it loads the full customer table into the agent's context and instructs the model, in words, to only discuss the current user's account — putting the entire security boundary inside a sentence the model can be talked past.

This is the pattern behind the leak. The security control is a polite instruction in the prompt. Everything the model can see, it can reveal, and it can see everyone's data. The only thing standing between a stranger and the whole database is the model's willingness to follow "only discuss this user's account" — a willingness any confused phrasing or crafted question can undermine.

Why It Fails

The weak version fails because it confuses instruction with enforcement, and those are not the same thing.

The model has access to data it should never touch. Loading the full table means every other customer's record is sitting right there in the context, one clever question away from the output. A prompt saying "don't look at the others" doesn't remove the data — it just asks the model not to mention what's already in front of it. Security by request, against a system designed to be helpful, loses.

There's no identity enforcement below the model. The code never checks whether this user is allowed to see this data — it delegates that entire decision to the language model's discretion. But authorization is a solved problem in every other part of software precisely because we don't leave it to discretion. Handing it to a probabilistic text model is a step backward disguised as a feature.

And the failure is silent and total. When it breaks, it doesn't error — it confidently returns the wrong customer's data as if that were the job. A healthcare or fintech team facing this isn't looking at a bug ticket; they're looking at a reportable breach, because regulated data left the boundary it was legally required to stay inside.

⚠️ Common mistake: Using the prompt as your access-control layer. "Only show this user their own data" is a hope, not a permission boundary. If the model can see the data, assume it can be made to reveal it — the sentence asking it not to is the weakest link in the system.

After: The Improved AI Agent Security Design

The strong version enforces access in code, before the model ever sees anything. The agent gets a narrow tool that only ever returns data the current user is authorized for.

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], ,[object Object], authz.can_access(session.identity, user):
        ,[object Object], SecurityError(,[object Object],)
    allowed = {,[object Object],, ,[object Object],, ,[object Object],}   ,[object Object],
    ,[object Object], field ,[object Object], ,[object Object], allowed:
        ,[object Object], SecurityError(,[object Object],)
    ,[object Object], db.query_one(user, field)

,[object Object], ,[object Object],(,[object Object],):
    tools = {,[object Object],: ,[object Object], field: get_account(field, session.user, session)}
    ,[object Object], agent.run(question, tools=tools)

What this does: it replaces the raw database connection with a scoped tool that checks the caller's identity and an allow-list of fields on every call — so the agent can only ever retrieve data the authenticated user is genuinely permitted to see.

Breaking Down Each Element of AI Agent Security

Four ideas turn a leaky agent into a contained one, and each closes a specific hole from the weak version.

The agent has no direct database access. It can't run queries; it can only call a tool that runs them for it, under rules the tool enforces. Removing the raw connection removes the possibility of the model reaching data outside its lane, no matter what it's told.

Identity is checked on every call, in code.

authz.can_access

The field allow-list caps what's exposable even for authorized data. The agent can see a plan name and a billing date but never another person's raw identifiers, because the tool refuses to return fields that aren't on the list. Least privilege applies to fields, not just records.

Failures are loud. An unauthorized access raises a real error that logs and alerts, instead of silently succeeding into the output. You find out at the attempt, not in a customer complaint weeks later.

⚡ Pro tip: Give the agent a service identity scoped to the acting user, not a superuser connection. When the agent authenticates as "acting on behalf of user X" rather than as an admin, your existing database permissions do the heavy lifting and an injection can't escalate beyond what X could already see.

Variations for Different Contexts

The design scales across sensitivity levels.

In healthcare, a clinical agent authenticates as the treating clinician and can reach only that clinician's patients, with every access written to an audit log the way regulations demand. The agent inherits the human's boundary exactly, and the log proves it.

In finance, an account agent exposes balances and transaction summaries through scoped tools but routes any action that moves money through a separate, human-gated path. Reading is scoped; acting is gated; neither depends on the model choosing to behave.

In internal tooling, an analytics agent queries only anonymized or aggregated views, never raw user rows, so even a fully compromised agent can't reveal an individual. When the agent's job is aggregate insight, it should never have row-level access in the first place.

⚡ Pro tip: Log every data access with the field, the acting identity, and the run ID. When you need to prove what an agent could and did reach — for an audit, an incident, or a regulator — a per-access log is the difference between a confident answer and a frightening shrug. Build it before you need it.

⚡ Pro tip: Assume the model will be tricked, and design so that being tricked isn't enough. Every control here holds even if the model fully believes it should hand over another customer's data — because the tool, not the model, decides what comes back. That's the test of real AI agent security: does it survive a model that's been completely fooled?

Keeping the Agent Useful While Locking It Down

The objection every team raises to tight scoping is that a locked-down agent is a useless one — if it can only see three fields, how does it answer real questions? The answer is that useful and unrestricted are not the same thing, and conflating them is what produces the leaky designs in the first place.

Start from the questions the agent genuinely needs to answer, then expose exactly the fields those questions require and nothing more. A support agent answering "when's my next bill and what plan am I on" needs the plan, the billing date, and the status — three fields — not the raw customer table. Nearly every agent turns out to need far less data than the easy design handed it. The full-table access wasn't serving the task; it was serving the developer's convenience, and it's precisely that surplus access an attacker exploits.

When an agent genuinely needs broader reach, scope it by relationship rather than by field. A clinical agent acting for a doctor should reach that doctor's patients — a real, meaningful set — but still not the whole hospital. The boundary follows the legitimate relationship the human already has, so the agent is as capable as the person it acts for and no more. A sales agent works the same way: it sees the accounts its rep owns, which is plenty to be useful and nowhere near the whole CRM.

Done this way, scoping doesn't cripple the agent. It aligns the agent's reach with the task's actual need, which is where it should have been all along. The agents that leak aren't the ones that were too restricted — they're the ones that were handed access no task ever asked for.

⚡ Pro tip: Derive each agent's permissions from a written list of the questions it must answer, not from what's convenient to connect. If a field or table doesn't map to a question on that list, it doesn't belong in the agent's reach. This one habit prevents the over-permissioning that turns a small manipulation into a large breach.

Save and Reuse This

The scoped-tool pattern, the identity check, and the field allow-list are the same shape for almost every agent that touches real data. Once you've built a data broker that enforces least privilege for one agent, the next one is a matter of swapping the fields and the authorization rule.

Keep these patterns where your team can grab them. Groups that store their security patterns and access-control templates in a shared library like PromptABCD ship each new agent with least privilege built in from the first commit, instead of bolting it on after the first scare. The agents you can trust with sensitive data aren't the ones with the strictest prompts — they're the ones that physically can't reach what they shouldn't.