Prompt Engineering for AI Agents

The Problem an Operations Lead Faced

An operations lead named Devon built an AI agent to help her team triage and route incoming vendor invoices, giving it access to a lookup tool and an approval-routing tool to handle the process end to end. During its first week live, the agent approved and routed an invoice from a vendor that didn't actually exist in the company's approved vendor list — it had looked similar enough to an approved vendor's name that the agent's lookup call returned a close match, and the agent treated that close match as confirmation rather than flagging the discrepancy to a human for review.

Nobody had told the agent what to do when a lookup returned a partial or approximate match instead of an exact one. This is the exact gap that prompt engineering for ai agents needs to close, and it's a fundamentally different problem than prompting for a single-turn text generation task — agents make sequences of decisions using tools, and each decision point needs its own explicit handling, not just a general instruction to "be careful" and hope that covers every situation that might come up.

⚠️ Common mistake: Giving an agent tool access and general instructions to "verify information carefully" without specifying exactly what counts as sufficient verification for each type of decision the agent might make. Vague carefulness instructions don't transfer into specific behavior at the exact decision points where it actually matters most.

The Wrong Approach

Devon's original agent prompt, written quickly during the initial build without much thought given to edge cases:

You have access to a vendor lookup tool and an invoice approval tool. When an invoice comes in, look up the vendor and approve if they're a valid vendor.

This gives the agent a task and tools but no guidance on what "valid vendor" actually means in edge cases — an exact name match, a fuzzy match, a match on some fields but not others entirely. The agent filled that gap with a reasonable-sounding but ultimately wrong interpretation: treating any returned lookup result as sufficient confirmation, rather than checking carefully whether it was actually an exact match on every relevant field.

The Correct Prompt

Here's the rebuilt agent prompt Devon now uses across her whole invoice-processing pipeline, with explicit handling for the specific failure that actually occurred in production:

You have access to: lookup_vendor(name), approve_invoice(vendor_id, amount).

For each invoice:
1. Call lookup_vendor with the exact vendor name from the invoice
2. Check whether the result is an EXACT match on vendor name and tax ID, not just a similar or partial match
3. If exact match: proceed to approve_invoice
4. If partial or fuzzy match only: do NOT approve. Flag for human review with a note explaining the discrepancy (e.g. "invoice vendor name 'Acme Corp' partially matches approved vendor 'Acme Corporation LLC' — confirm before approving")
5. If no match at all: flag for human review as a potentially new or unapproved vendor

State your reasoning (Thought) before each tool call (Action), and record what each tool actually returned (Observation) before deciding the next step.

What this does: explicitly defining what counts as a match, rather than leaving "valid vendor" as an interpretive judgment call left up to the model, closes the exact gap that caused the original failure. The Thought/Action/Observation structure, borrowed from the ReAct pattern, makes each decision auditable after the fact, which matters enormously for a financial process where mistakes have real cost attached to them.

⚡ Pro tip: For any agent making decisions with real consequences, explicitly define what counts as sufficient evidence for each type of decision it might face, rather than leaving that threshold to the model's own judgment in the moment. "Valid" or "verified" are judgment calls that need concrete criteria attached, not vague adjectives left open to interpretation by whoever's reasoning through the decision.

⚡ Pro tip: When writing decision criteria for an agent, phrase them as concrete checks rather than qualities to weigh. "Exact match on name and tax ID" is a check the model can apply consistently; "reasonably confident this is the right vendor" is a quality judgment that will vary from run to run depending on subtle context.

Results and What Changed

After the rebuild, Devon ran the agent against three months of historical invoice data to check for other cases it would have mishandled under the old logic before the fix. It found two additional partial-match cases that had actually been approved incorrectly under the old process, which the new logic would have correctly flagged for human review instead of silently approving them and letting them through unnoticed.

A procurement lead at a different company building a similar agent added an additional layer: a hard dollar-amount threshold above which the agent always flags for human review regardless of how confident its vendor match was, since some decisions warrant human oversight regardless of how well-verified the underlying facts appear to be to the agent's own reasoning process.

⚡ Pro tip: Combine specific decision-point guidance with hard thresholds for your highest-consequence actions. Good reasoning instructions reduce errors generally, but a hard rule — like a dollar threshold requiring human sign-off no matter what the agent concludes — provides a backstop that doesn't depend on the reasoning working correctly every single time it's tested.

⚡ Pro tip: Test any agent against historical data before trusting it on live decisions that carry real consequences. Devon's three-month historical test surfaced problems that hadn't yet caused visible harm in live operation but would have eventually, given enough volume and enough time for an edge case to actually occur in practice.

How to Apply This to Your Situation

Before deploying any agent that makes decisions using tools, map out the actual decision points carefully — not just the overall task, but each specific fork where the agent has to judge something as sufficient, valid, or safe to proceed. For each fork, write explicit criteria rather than trusting a general instruction to cover it adequately across every situation it might encounter down the line.

⚠️ Common mistake: Testing an agent only against clean, typical cases before deploying it live into production. The failure modes that actually matter tend to live in edge cases — partial matches, ambiguous inputs, unusual combinations — precisely because those are the situations vague instructions don't cover well and general carefulness doesn't reliably catch, no matter how many times you re-read the prompt before shipping it.

It's worth involving whoever handled these decisions manually before the agent existed in this mapping exercise, since they usually have implicit knowledge about edge cases that never got written down anywhere because they'd just handle them by instinct in the moment. Devon's own vendor-matching gap only became obvious once she sat down with the accounts payable clerk who used to do this manually and asked specifically what she'd do if a vendor name looked "close but not quite right" — a scenario that had never come up explicitly in any conversation about building the agent until that exact point in the process.

Next Steps

Build a habit of running any new or modified agent against historical data before trusting it live in production, the same way Devon now does for every change to her invoice-routing agent. This surfaces latent problems while they're still cheap to fix, rather than after they've caused a real, possibly costly mistake in production that someone else has to clean up after the fact.

Devon also instituted a lighter-weight version of this check for smaller agent modifications going forward: instead of a full three-month historical replay every time, she runs any prompt change against a curated set of twenty known edge cases she's accumulated over time, covering every failure mode she's discovered so far in production. This catches regressions fast without requiring a full historical replay for every minor tweak to the agent's instructions.

There's a broader principle worth carrying forward from Devon's experience: agents that make consequential decisions deserve the same kind of edge-case rigor that engineers apply to testing code, not the more casual, conversational testing that's often good enough for a single text-generation prompt someone just uses once. The stakes are simply different when an agent is taking real actions with real consequences rather than just drafting text for a human to review before anything actually happens.

A finance team lead who adopted this rigor after hearing about Devon's near-miss now requires any new agent with financial decision-making authority to pass a documented edge-case test suite before it's allowed to operate without a human in the loop for every single decision it makes. This isn't bureaucracy for its own sake — it's the direct lesson from exactly the kind of gap that let an unapproved vendor's invoice nearly slip through undetected in the first place.

Once you've built decision-point criteria that handle your agent's specific edge cases well, document them alongside the agent's core prompt, not as a separate afterthought that's easy to lose track of over time. PromptABCD is useful here for keeping an agent's core instructions and its edge-case handling versioned together, so the next person maintaining the agent inherits the full picture of what's been learned, rather than rediscovering Devon's fuzzy-match lesson the same expensive way she did the first time.