Entity Extraction with AI: Prompts That Work

A 2025 study from MIT found that AI-assisted entity extraction outperformed manual extraction by 34% on accuracy -- but only when the prompts were specifically designed for extraction tasks. Generic prompts applied to extraction work produced results 18% worse than manual review. The gap between a well-designed entity extraction ai prompt and a generic one is not small.

This guide covers the prompt structures that actually work for extracting structured information from unstructured text -- across industries and document types.

Before: The Weak Prompt

Here's the extraction prompt most people write first:

Extract the key entities from the following text.

[document text]

What this does: Asks for "key entities" without defining what entities are relevant, what format the output should take, how to handle ambiguity, or what to do when information is missing. The model fills every gap with its own defaults -- which will be different every run and often wrong for your use case.

Why It Fails

Entity extraction prompts fail for four specific reasons:

Undefined entity schema: Without knowing which entity types to extract (person, organization, date, amount, location, product), the model extracts whatever it considers "key" -- which is subjective and inconsistent across documents.

No handling for missing fields: When a document doesn't contain a required entity type, the model either invents a plausible value (hallucination) or omits the field entirely. Without explicit instructions, you get both behaviors randomly.

No format anchoring: "Extract entities" produces prose, bullet lists, JSON, or tables depending on the model's mood. Downstream systems can't reliably parse inconsistent output formats.

No confidence handling: Some entities are unambiguous. Others are ambiguous. Without instructions for handling ambiguity, the model picks one without flagging the uncertainty.

⚠️ Common mistake: Testing extraction prompts on clean, well-formatted documents and deploying to production where documents are messy, scanned, or inconsistently formatted. Always test on your worst-case real documents, not your best examples.

After: The Improved Prompt

You are a structured data extraction specialist. Extract the following entity types from the document below.

Extract ONLY these entity types:
- vendor_name: string (the company issuing this invoice)
- invoice_number: string (exact as written, including any prefix)
- invoice_date: string (format as YYYY-MM-DD if possible; use original format if ambiguous)
- due_date: string (format as YYYY-MM-DD; null if not present)
- line_items: array of {description: string, quantity: number, unit_price: number, total: number}
- total_amount: number (grand total only, excluding line items)
- currency: string (3-letter ISO code; default USD if not specified)

Rules:
- If a field is not present in the document, use null -- never guess or infer
- If a field is ambiguous (two possible values), include both in an array and add: ambiguous: true
- Output ONLY valid JSON. No explanation, no preamble.

Document:
[document text]

What this does: Defines every entity type with its expected data type, provides explicit null-handling rules, specifies ambiguity behavior, and enforces JSON-only output. Each rule targets a specific failure mode from the Why It Fails section.

Breaking Down Each Element

The entity schema is the most important element. Not just "extract dates" -- extract this specific date field in this specific format as this data type. Every entity type should have: name, data type, and at least one disambiguation note.

Null rules prevent hallucination. "If a field is not present, use null -- never guess or infer" is the single most important rule in any extraction prompt. Without it, the model fills missing fields with statistically plausible values -- which look correct and are often wrong.

Ambiguity handling separates professional extraction from amateur extraction. Real documents have ambiguous fields. A prompt that doesn't handle ambiguity forces the model to make undisclosed judgment calls. Surfacing ambiguity explicitly lets downstream systems route uncertain records for human review.

⚡ Pro tip: For high-volume extraction pipelines, add a validation step after extraction. Pass the JSON output to a second prompt that checks schema compliance, flags null fields above a threshold, and identifies records marked ambiguous. This two-stage approach catches errors before they reach downstream systems.

Variations for Different Contexts

For medical records extraction (patient name, diagnosis codes, medication dosages, dates of service):

CRITICAL: This extraction task involves medical data. Apply these additional rules:
- Never infer or estimate medication dosages -- extract exactly as written or use null
- Dates must include time zone if present in original
- If a diagnosis code appears without an ICD version indicator, flag as: icd_version_unknown: true
- Any field where extraction confidence is below very high should be flagged with: review_required: true

What this does: Adds domain-specific safety constraints on top of the base extraction template -- ensuring that medical extraction errors are surfaced rather than silently passed through.

For legal document extraction (parties, dates, obligations, consideration amounts): Add "When a clause can be interpreted in more than one way, extract both interpretations as an array and flag: legally_ambiguous: true. Do not resolve legal ambiguity -- preserve it."

⚡ Pro tip: Build entity schemas as reusable modules. A "person entity" schema (name, role, contact) is the same whether you're extracting from contracts, invoices, or emails. Maintain a schema library and compose extraction prompts by combining relevant schemas rather than writing from scratch each time.

Industry Use Cases

Healthcare: Clinical notes extraction -- patient vitals, medication orders, diagnosis codes, follow-up instructions. Accuracy requirements are highest here; the null-rule and ambiguity-flagging are non-negotiable.

Financial services: Invoice processing, contract term extraction, transaction categorization. The volume is high enough that even 2% error rates are expensive at scale.

Legal: Contract clause extraction, obligation identification, deadline extraction. The ambiguity handling pattern is especially critical -- legal language is often intentionally ambiguous.

⚡ Pro tip: For any extraction task running in production, calculate your error rate on a sample of 50-100 documents per month. Extraction accuracy degrades over time as document formats change and model behavior shifts. Catching a regression early is far cheaper than discovering it after thousands of misprocessed records.

Measuring Extraction Quality

Most teams evaluate extraction by checking whether the output looks right on a few sample documents. That's not enough for production systems. Here's a more rigorous approach:

Extraction quality evaluation protocol:
1. Sample 30 documents from production (not test data)
2. Have a human expert extract the same fields independently
3. Compare AI extraction to human extraction field by field
4. Calculate per-field accuracy, not just overall accuracy
5. Identify which fields have accuracy below 90% -- these need targeted prompt improvements
6. Re-run the evaluation monthly to catch model behavior drift

What this does: Identifies weak fields specifically rather than averaging problems away. A prompt that is 99% accurate on 5 of 6 fields and 40% accurate on the 6th looks like 91% overall -- but that 6th field is a critical problem that averaging obscures.

⚡ Pro tip: Weight your accuracy calculation by the business impact of each field. A wrong invoice number is catastrophic (payment goes to wrong vendor). A wrong currency code is serious. A wrong line item description is minor. Per-field accuracy weighted by business impact gives you the right improvement priorities.

Save and Reuse This

Entity extraction prompts are among the most valuable reusable assets in any AI-assisted workflow because they encode domain knowledge (the schema) alongside the task instructions. The schema you define for invoice extraction reflects months of edge-case experience -- don't rebuild it from scratch every time.

Save your entity extraction templates with their schemas in PromptABCD. Version them when schemas change. When a new document type appears, start from your closest existing schema and adapt.

Common Mistakes Beyond the Basics

The null-rule and schema discipline eliminate most extraction failures. But three subtler problems persist even in well-structured extraction prompts:

Nested entity confusion: When extracting line items from an invoice (each with its own amount, description, and quantity), the model sometimes flattens the nesting -- outputting a list of amounts rather than an array of objects. Fix this by showing a minimal JSON example of the correct nested structure in the prompt itself.

Expected output format for line_items:
"line_items": [
  {"description": "Consulting - March", "quantity": 10, "unit_price": 150.00, "total": 1500.00},
  {"description": "Expenses", "quantity": 1, "unit_price": 47.50, "total": 47.50}
]
Not this: "line_items": ["Consulting - March", "Expenses"]

What this does: Shows the model the exact nested structure required, including a "not this" anti-example -- which is often more useful than showing only the correct version.

Date format normalization drift: Even with explicit date format instructions, models occasionally output dates in inconsistent formats when documents contain multiple different date styles. Add: "If the same document contains dates in multiple formats, normalize ALL dates to YYYY-MM-DD regardless of their source format." The explicit "regardless" overrides the model's tendency to preserve source formatting.

Currency ambiguity in international documents: Documents from companies operating in multiple markets sometimes contain amounts in multiple currencies without clear labeling. Add a tiebreaker: "If currency is ambiguous, use the currency of the vendor's registered country, not the buyer's country."

⚡ Pro tip: After building an extraction prompt, run it on 10 documents from your collection and manually review every null output. Nulls reveal two things: fields the model genuinely couldn't find (correct behavior) and fields the model missed due to formatting variations (prompt improvement opportunity). The ratio tells you how well your extraction is actually performing.

⚡ Pro tip: PromptABCD is especially valuable for extraction prompt libraries because extraction schemas change as document formats evolve. Version your schemas, note which document types each version handles, and maintain a changelog. When a new vendor format breaks your extraction, you'll have the old schema to compare against.