Information extraction transforms unstructured document content into structured
data. As of config v0.6, per-field confidence and geometry (bounding
boxes) are treated as outputs of extraction, not a separate downstream
stage. Their settings live under extraction.confidence.* and
extraction.geometry.*; human-in-the-loop review is configured under the
top-level hitl.* block.
This guide consolidates what used to live across three separate documents
(Customizing Extraction, the Assessment Feature, and Bounding Box Integration)
into one place, reflecting the v0.6 model where extraction produces the value,
its confidence, and its location together.
Extraction mode — how values are pulled from the document: Simple (one inference) or Advanced/agentic (a tool-using agent that can shard, validate, and self-correct).
Confidence mode — how per-field confidence + bounding boxes are produced: separate (a dedicated confidence pass), integrated (confidence rides on the extraction inference), or off.
They combine freely. The tables below give pros/cons and a recommendation; the rest of the guide is the detail.
One Bedrock inference returns the structured result
Strands agent with a structured-output tool; can shard large sections, validate against the schema, and self-correct
Pros
Cheapest & fastest; fewest moving parts; works with every model incl. OpenAI GPT-5.x
Highest accuracy on complex/nested schemas; guaranteed schema compliance; deterministic table parsing for big tables; sharding for long docs; model escalation on validation failure
Cons
No built-in validation/retry; a single huge document must fit one inference (context-overflow / read-timeout risk); weaker on deeply nested structures
More inferences → higher per-doc cost & latency; requires a model that reliably emits tool use (no OpenAI GPT-5.x, and not Amazon Nova Lite — see below); still in preview
Choose when
Most documents; small–medium size; simple/flat schemas; lowest cost matters
Complex/nested schemas, strict validation needs, large documents or big multi-row tables, business-critical accuracy
Rule of thumb: start Simple. Move to Advanced when you hit nested-schema accuracy limits, need schema-format validation, or the document is large enough that one inference can’t hold it (long tables, 20+ dense pages).
A dedicated confidence inference (Simple: the standalone Assessment step; Advanced: one pass inside each shard)
Confidence rides on the extraction inference — no separate pass. Simple uses 1S-TopK (the model returns top-K guesses + probabilities per field); Advanced has the agent emit confidence via its tools
No confidence produced
Inferences added
+1 (Simple) / +1 per shard (Advanced)
0 (Simple, single-shot) / 0–1 (Advanced)
0
Pros
Best-calibrated (a fresh look at finalized values); can use a cheaper model than extraction (default Nova Lite); large lists batched reliably
Fewest inferences → lowest confidence cost; one round-trip on the simple path
Zero confidence cost
Cons
Extra inference(s) → more cost/latency than integrated
On the Simple path everything (context + values + all confidence) must fit one response → truncation risk on large docs/tables; can’t use a separate cheaper model
No confidence → no HITL routing, no UI confidence/threshold signals
Choose when
Default for almost everyone — the calibration and cheap-model economics usually win
Small documents where one inference comfortably holds values and confidence, and you want to minimize round-trips
Confidence genuinely not needed (no HITL, no reliability signal)
Recommended defaults:Simple + separate for most workloads; Advanced + separate for complex or large documents. Reach for integrated only on small docs where minimizing inferences matters, and off only when you don’t consume confidence at all.
Simple + integrated uses 1S-TopK. On the Simple path, integrated mode asks the model for its top-K guesses with probabilities per field (G1/P1 … GK/PK) in one call; the top guess becomes the value and its probability the confidence. Enumerating alternatives yields better-calibrated, less-overconfident scores than a single value + a single confidence number (Tian et al., “Just Ask for Calibration”, EMNLP 2023). The output result.json is identical in shape to separate mode (same inference_result + explainability_info), so HITL, evaluation, reporting, and the UI are unchanged, and the standalone Assessment step auto-skips — for scalar-only classes; a list-bearing class is scored separately regardless (see the warning below). The prompt is editable in the UI (“Task prompt (1-Stage TopK extraction + confidence — simple)”) / extraction.task_prompt_extraction_with_confidence_topk. See the reference config and the extraction library README.
⚠️ Simple + integrated is automatically scored in a separate pass for list-bearing classes. On a class that declares any list field (including every multi-instance class, whose instances array is a list), a Simple-mode section ignores integrated and runs the standalone Assessment step instead. Its cost relative to the single inference is model-dependent: at the shipped default model (Claude Sonnet 5) a 100-row statement cost about 2.5× ($0.608 vs $0.247, batched list scoring), while at Claude Sonnet 4.6 the same cell was cheaper than the integrated call in the 2026-09-09 live pass ($0.131 vs $0.171). Either way the result is complete: Simple + integrated returned 10 of 100 rows while Simple + separate returned 100 of 100 in every repeat. The mechanism is output volume: the TopK envelope costs several guesses per cell, so a list that fits comfortably in a plain extraction can exceed what the model will emit in one response, and it stops emitting rows rather than erroring. Two changes reduce it — list cells are asked for a single guess (G1/P1 only) instead of four, and the prompt no longer tells the model to make each guess “as short as possible” (which also shortened values) — but the single-response limit remains. Scalar-only classes keep the single-inference saving, and Advanced extraction is unchanged (sharding keeps each call small). Two per-class opt-outs keep a list-bearing class on 1S-TopK: its own x-aws-idp-extraction-task-prompt (the downgrade works by swapping the prompt, so a user-controlled prompt is never half-applied), or the explicit x-aws-idp-allow-integrated-lists: true flag for a class whose lists you have verified come back complete on your own documents — the 1S-TopK reference config sets it on its Invoice class for that reason. Where you see the decision: idp-cli config-validate (and the SDK validate operation) warns and names the affected classes — the web UI does not validate on save, but its Prompt Preview pane shows, per class, that the plain prompt is sent; per document, metadata.confidence_mode_effective: separate and the Processing Flow in the Processing Report record it. It is deliberately not a Processing Issue, because that flag is severity-blind and would badge every document. See config-guidance for the measured comparison.
Large lists & truncation are handled on every path. Whichever combination you pick, long list fields are assessed in sequential batches (list_batch_size), and if the confidence model truncates a batch at its output-token ceiling the batch is recursively split until it fits — so you get complete per-cell coverage without tuning. See Large-list batching. For documents that are large because of a very large single section (not just a long list), prefer Advanced sharding — see Large-Document Guidance.
Simple extraction sends the system prompt and task prompt to Bedrock in a single
inference and parses the structured (JSON or YAML) response. It is the default
and is a good fit for the majority of documents.
extraction:
agentic:
enabled: false# Simple mode (default)
model: anthropic.claude-haiku-4-5-20251001-v1:0
temperature: 0.0
reasoning_effort: low# reasoning-capable models only (see note below)
Output tokens: extraction and the confidence pass by default request the
selected model’s maximum output — there is no max_tokens config knob for
them. Bedrock’s default-when-omitted truncates, so the client sets it
explicitly from the per-model limits (seeded from
config_library/model_config_limits.yaml and editable in the web UI under
View / Edit Model Limits); completeness matters more than an output cap
here. The one exception is a confidence call on Amazon Nova Lite/Micro, which
requests an output budget sized to its rows (see the self-healing note under
Large-list batching) because that model
was measured looping to its cap. (classification / summarization keep their
max_tokens knob.)
Reasoning effort: for reasoning-capable models — Claude Sonnet 5 / Sonnet
4.6 / Opus 4.5–4.8 / Opus 5 / Opus 5.5 / Fable 5 / Haiku 5.5
(low|medium|high|xhigh|max), OpenAI
GPT-5.x (minimal|low|medium|high), xAI Grok
(none|low|medium|high|xhigh, notmax), and OpenAI GPT-6 Astra
(none|low|medium|high|xhigh|max, notminimal) — reasoning_effort controls how much
the model reasons before answering. Extraction defaults to low: a full
effort sweep found higher effort adds output-token cost with negligible
extraction-accuracy gain. Raise it per-config for reasoning-heavy documents.
Ignored by Nova, Sonnet 4.5, and Haiku 4.5 — note the two Haikus differ here:
Haiku 5.5 is the first Haiku that accepts effort, and its own default is
medium rather than high.
Advanced (agentic) extraction uses the Strands agent framework with tools for
structured output, giving superior accuracy and consistency — especially for
complex documents with nested structures or strict schema requirements. It
shards large sections and runs both extraction and confidence assessment
per shard.
Preview Status: Agentic extraction is currently in preview. While it
demonstrates significant improvements in accuracy and reliability, we
recommend thorough testing in your specific use case before production
deployment.
When to enable agentic extraction — when you need:
Schema Compliance: Guaranteed adherence to defined data structures
Data Validation: Automatic validation with retry mechanisms
Complex Structures: Proper handling of nested objects and arrays
Date Standardization: Consistent date formatting
Self-Correction: Automatic fixing of extraction errors
Production Reliability: Higher accuracy for business-critical data
Extensibility: Future integration with Model Context Protocol (MCP) servers for advanced validation, enrichment, and external data lookups during extraction
Agentic extraction requires models with tool-use support:
Anthropic Claude Sonnet models (recommended for optimal performance)
us.anthropic.claude-sonnet-5 — the shipped extraction default
us.anthropic.claude-sonnet-4-6 — best balance of speed and cost
us.anthropic.claude-sonnet-4-5-20250929-v1:0 — still selectable, but an AWS
Health notice puts it in legacy with end of life on 2027-04-08
Anthropic Claude Opus models (for highest accuracy requirements), e.g.
us.anthropic.claude-opus-5
Amazon Nova Pro (AWS native alternative) — no successful agentic run has
been measured for Nova Pro: its advanced cells in the v0.6.8 sweep hit the same
mid-stream tool-use failure described below and the grid was abandoned, so it is
unmeasured on this path rather than known to be incapable. Treat it as unproven
until a benchmark run completes on it
⚠️ Amazon Nova Lite does not complete Advanced (agentic) extraction as
shipped. On the agentic path Nova Lite fails mid-stream with Bedrock’s
modelStreamErrorException: Model produced invalid sequence as part of ToolUse
— the model emits a toolUse block the protocol rejects. The v0.6.8 benchmark
refresh logged 247 of these in one three-hour window across 40 shard-runtime log
streams, all of them us.amazon.nova-lite-v1:0
(#895).
The failure reproduces on retry with the same request, so since v0.6.9 it is
classified as deterministic: the shard fails in seconds with a message naming
the model and the remedies instead of being retried by the state machine.
Greedy decoding does not help — that is measured. AWS’s
Nova tool-use troubleshooting guide
— the guide Bedrock’s own error message points at — attributes this error primarily
to inference parameters rather than to raw model capability, and recommends greedy
decoding: temperature: 0andtopP: 1andtopK: 1. All three were sent
and the failure rate did not move
(#956).
In 160 direct Bedrock calls per arm, interleaved so capacity drift hit both arms
equally, Nova Lite emitted an invalid tool-use sequence in 0.787 of calls with
no topK, 0.812 with topK: 1, and 0.838 with the full greedy triple
(Fisher exact p = 0.68 for topK: 1 against the pooled no-topK arm, and p = 1.00
for the greedy triple against the no-topK arm measured in the same batch). Every
point estimate moved in the unhelpful direction, none of them significantly.
⚠️ Read that as “no large effect”, not as “proven identical”. At the observed
base rate and n = 160 per arm, 80% power detects an improvement only down to a
failure rate of 0.65 — 18% relative. Ruling out a 10-point absolute improvement
would need ~303 calls per arm and a 5-point one ~1,140. A small benefit is not
excluded; a large one is.
topK is also not a setting you can simply turn on, which is why the
accelerator exposes no option for it. inferenceConfig.topK is not a Converse
field at all — botocore rejects it client-side for every model — so it can only
travel in additionalModelRequestFields, in a shape that differs per provider:
Family
Carrier that works
Valid range
Nova Lite / Pro / 2 Lite
additionalModelRequestFields.inferenceConfig.topK
1–128
Claude ≤ 4.6 (Haiku 4.5, Sonnet 4.5, Sonnet 4.6, Opus 4.5)
additionalModelRequestFields.top_k
−1 – 100,000,000
Claude 4.7+ (Opus 4.7, Opus 4.8, Opus 5, Opus 5.5, Sonnet 5, Haiku 5.5)
none — `top_k` is deprecated for this model
—
OpenAI GPT-6 Astra
none — Unknown parameter: 'top_k'
—
xAI Grok 4.6
unverifiable — returns 200 for any unknown key, including a deliberately bogus control
—
Each family rejects the other’s spelling, so there is no single request shape that
would set it across the models offered here.
What does trigger it is the request shape, not its size. Walking the request
from trivial to real on Nova Lite: a two-property scalar schema succeeds; a schema
holding only a transactions array succeeds and returns all 30 rows, at both 284
and 1,696 characters; the same array plus three sibling scalar properties — 401
characters, smaller than the one that succeeded — fails. Removing format,
pattern and anyOf, inlining $defs, and removing the system-prompt schema
restatement each changed nothing; removing the document text made it pass. So the
trigger is a list-of-objects property that must be closed before further
top-level keys are emitted, with real content to fill it — which is the shape of
almost every useful extraction class. Whether that is the only triggering shape is
open: one variant with no array in the schema at all also failed, under a prompt
that demanded rows the schema could not hold, so it is confounded. Full tables,
sample sizes and the power analysis:
Greedy decoding and invalid tool-use sequences.
Failing fast remains the right behaviour: the failure recurs on every attempt with
the same request, and no inference-parameter setting available on this model changes
that.
Remedies, in the order worth trying: reduce
extraction.agentic.shard_token_budget / extraction.agentic.max_pages_per_shard
so each call emits less; set extraction.model to a model measured on this path
(Claude Sonnet 5, Sonnet 4.6, Opus 5, OpenAI GPT-6 Astra, xAI Grok 4.6); or use
extraction.mode: simple.
This is specific to Advanced extraction. Nova Lite remains fully supported —
and is the shipped default — for the confidence-assessment pass
(extraction.confidence), and it works for Simple extraction, which needs no
tool use in its default configuration (the experimental forced_tool path is the
one exception). See
Benchmarking → which models are measured.
⚠️ OpenAI GPT-5.x cannot be used with agentic extraction. All
openai.gpt-5.* models (openai.gpt-5.4, openai.gpt-5.5, and GPT-5.6
Sol/Terra/Luna) run on the bedrock-mantle Responses API and do not support
the Converse-based Strands agent loop. Pairing an OpenAI model with
extraction.agentic.enabled: true is a hard error in
idp-cli config-validate and raises at runtime. OpenAI models are fully
supported for Simple (non-agentic) extraction. See
OpenAI GPT-5.x Models.
✅ xAI Grok CAN be used with agentic extraction. Grok 4.6
(us.xai.grok-4.6, global.xai.grok-4.6) is served on the standard Converse
API and accepts a toolConfig with all three toolChoice modes, so the Strands
agent loop works — so alongside Anthropic Claude and OpenAI GPT-6 Astra it is one
of the families measured on the Advanced path. Its 500K context also yields a
larger shard budget (~90K tokens)
than a 200K-context Claude model (~18K). Note that temperature / top_p are
rejected by Grok and silently omitted; tune it with reasoning_effort
(none|low|medium|high|xhigh) instead. See
xAI Grok Models.
✅ OpenAI GPT-6 Astra CAN be used with agentic extraction. Unlike the
GPT-5.x models above, Astra (us.openai.gpt-6-astra,
global.openai.gpt-6-astra) is served on the standard Converse API and emits a
toolUse block under a forced toolChoice, so the Strands agent loop works.
The agentic gate keys on the route, not the vendor prefix — which is why one
OpenAI model is allowed here and the others are not. Its 1.05M context yields
the largest shard budget of any model offered. temperature / top_p are
rejected and silently omitted; tune it with reasoning_effort
(none|low|medium|high|xhigh|max) instead. Watch cost: Astra is the
priciest model available, and input above 272K tokens bills at roughly double
the rate recorded in pricing.yaml. See
OpenAI Models.
Agentic extraction may have slightly higher costs due to additional processing
for validation and correction, tool-use requiring more capable models, and
retry attempts. The benefits typically outweigh the costs: agentic extraction
improves model performance significantly — for example, Claude Sonnet 3.5 gains
over 20% in accuracy on the getomni-ai benchmark.
100% schema compliance vs frequent validation failures
Reduced manual review and correction efforts
Automatic caching: For supported models, prompt and tool caching is automatically enabled, reducing costs for repeated extractions with the same configuration
Dropping the duplicated schema (restate_schema_in_system_prompt)
Agentic extraction sends your class schema three times per request: as the
tool’s input schema (required by the API), restated as JSON in the system prompt,
and again in the schema-reminder tool. Copies 2 and 3 are byte-identical, so the
system-prompt copy is pure duplication — measured at 2,600 of 6,680 schema
tokens (38%) on the lending Payslip class.
Left on by default deliberately. Restating a schema in prose often improves how
closely a model follows it, so this is a token/adherence trade rather than a free
saving — an agent that drifts from the schema on a list-heavy document returns fewer
rows. No such loss appeared in the 50-run measurement below, but that measurement used
one model and three synthetic documents, so on your own documents judge the result on
completeness, not on the token count. The schema-reminder tool is unaffected either
way, so the agent can always ask for the schema again mid-run.
What to expect if you turn it off: fewer input tokens, no quality change, and a
slightly higher bill. Measured over 50 runs, 25 per arm, on Sonnet 4.6 across
three synthetic documents
(full study):
Quality: no cost.completeness_recall, cell_accuracy and
scalar_accuracy were 1.000 in every one of the 50 runs in both arms, with no
failures and identical cells_compared. Both arms sit on the ceiling, so this
excludes a degradation affecting more than roughly one run in ten; it cannot show
an improvement.
Input tokens: a real saving, on the documents where the agent loop is stable.
−5.2% on a 400-row statement (175,912 → 166,731 input-side tokens, p = 1.3 ×
10⁻⁸) and −6.7% on a 100-row one (p = 0.008), with the arms completely
separated. On the third document the point estimate went the other way by +9.5%
and was not significant, because that document’s turn count varied run to run and
each turn re-reads the prefix.
Dollars: not a saving. Extraction cost’s central estimate rose 7.5% and
total cost 5.3% with the restatement off (neither significant, p ≥ 0.13),
because extraction output tokens rose 17.4% (p = 1.2 × 10⁻⁵). The
arithmetic is one-sided: the input saving is almost all cache reads at a tenth
of input price, worth ~$0.007 per document, while the extra output is worth
~$0.039.
So the reason to use this knob is shard headroom, not cost. Since
#775
the pipeline subtracts the measured prompt overhead (system prompt, rendered schema,
few-shot text, tool schema, this restatement) when it decides how to split a
document, where before the schema text was not counted at all — so the reclaimed
tokens are real budget. Whether the freed headroom changes a given document’s shard
count depends on it sitting near a boundary, and the page ceiling still closes shards
regardless.
Visible in the Prompt Preview. With Extraction mode Advanced, the
Configuration → Prompt Preview → System Prompt tab ends with the
Expected Schema: block this setting controls, and it is included in the token
total — so turning the setting off shows the total drop instead of leaving it
unchanged. One caveat the preview states rather than hides: the real block is
generated from a Pydantic model built from your class (every field gains a
title, and every optional field — plus, in Advanced mode, every required
scalar, so the agent can leave an unreadable cell null rather than invent a
value — an anyOf with {"type": "null"}), so the browser can only approximate
it from the class schema. Cells the agent left null because it could not read them
are counted under metadata.abstained_fields (scalar cells only — a whole list
returned as null is a defect, not an abstention) and shown in the Processing
Report, regardless of whether schema validation is enabled. Expect the real block to be
larger than the preview’s — roughly 1.7–2.9x on the shipped lending presets —
which makes the previewed saving a floor, not a ceiling.
How agentic uses your configuration: Agentic extraction automatically
converts your document class configuration (classes, attributes, descriptions,
types) into Pydantic models internally. Improving your configuration directly
improves extraction accuracy. A future enhancement will let you define custom
Pydantic models with validators, custom types, and business logic — including
MCP server integration for real-time external lookups.
For documents with large tables (bank statements, brokerage holdings,
transaction logs), enable the deterministic table parsing tool so the agent
extracts tabular data by parsing Markdown tables directly from OCR — instead of
having the LLM regenerate every row (slow, costly, and error-prone for hundreds
of rows):
extraction:
agentic:
enabled: true
table_parsing:
enabled: true# give the agent a deterministic parse_table tool
min_parse_success_rate: 0.90# below this, fall back to LLM extraction
max_empty_line_gap: 3# tolerate page-break gaps inside a table
auto_merge_adjacent_tables: true
lazy_images: true# skip pre-loading page images when the table
# parse succeeds (big cost saver; see below)
lazy_images (default true) — cost optimization for table documents. When
a pre-flight table parse succeeds, page images are not attached to the
extraction prompt. The deterministic table tool is text/markdown-driven and never
reads images, and the agent can still fetch a specific page on demand via the
view_image tool. Because the agentic loop re-sends the prompt on every turn,
pre-loaded images are re-transmitted repeatedly and dominate cost on multi-page
documents (and push large documents toward context-window limits). A controlled
A/B measured no change in list completeness or field accuracy with images off on
the table path. Set lazy_images: false for image-dependent corpora where the
model must see page layout/marks even when a table is present.
The agent is told what was already parsed. The pre-flight parse happens
before the agent runs, so its instructions gain a PRE-PARSED TABLE DATA AVAILABLE block: how many tables were found, how many rows in total, the column
list, what the --- PAGE N --- markers mean, and the parse_table →
map_table_to_schema → finalize_table_extraction sequence — ending with the
point that finalize reads the mapped rows from the agent’s own state, so it
never needs to write the rows out itself. That last part is what keeps output
tokens (and cost) down: an agent that does not know the rows exist re-emits them
one by one. Both the single-pass and the sharded path send this block; on the
sharded path each shard agent also gets its own concrete page range, plus a note
that the table and row totals in the block are for the whole section rather than
for its own pages — so parsing fewer rows than the total is the correct outcome
for a shard, not an incomplete one. Sharding is
the default for any multi-page table document, and until
issue #900
its agents were the only ones not receiving the block.
Requires Markdown tables in the OCR output. Table parsing only engages when
OCR emits Markdown pipe-tables — i.e. Amazon Textract with the TABLES
feature enabled (keep LAYOUT + TABLES in ocr.features), or another OCR
backend that produces Markdown tables. With plain-text OCR the agent falls back
to LLM extraction (use sharding, below, for large tables in that case).
Agentic extraction can validate its output against the full class JSON
Schema — including format keywords (date, email, uuid, …) that
type-checking alone misses — and, on failure, escalate the failing fields to a
stronger model:
escalation_model: "us.anthropic.claude-opus-4-8"# stronger tier; blank = retry same model
min_population_ratio: 0.5# advisory: warn if <50% of fields populated (silent-loss guard)
fail_action: escalate re-extracts only the failing top-level fields with escalation_model and merges them back — far cheaper than human review. The re-extraction replaces the original only if it lost no populated data (a list that had rows must not come back null or shorter; a filled value must not come back null) and got better field by field; a result that merely has fewer errors in total is not enough, because nulling a whole 100-row list produces one error where 100 unreadable cells produce 100. The decision and its reason appear in the Processing Report as escalation_kept / escalation_decision. warn records the outcome and proceeds; reject records the result as not parsed, which makes the section’s Processing Report read FAILED and raises the issue at error severity — it does not change the section’s or the document’s outcome, and it does not route the section to human review (HITL routing is driven by hitl.confidence_threshold, not by this).
A per-class override x-aws-idp-extraction-escalation-model takes precedence over the global escalation_model.
min_population_ratio is an advisory completeness heuristic: it flags suspiciously sparse results (e.g. a table that returned zero rows) without failing extraction.
Outcomes are recorded per section under metadata.validation and metadata.population_check, and surfaced in the Web UI Processing Report tab.
format: date caveat. JSON-Schema format: date means ISO-8601
(YYYY-MM-DD). The default extraction prompt asks the model for MM/DD/YYYY,
which will fail format validation — set check_formats: false or use a
pattern for non-ISO dates.
Scalable extraction for large documents (sharding)
For long or dense documents, agentic extraction shards a section’s pages into
token- and page-budgeted ranges and extracts them concurrently, then merges the
results (list rows concatenated in page order; scalars resolved first-non-null).
This bounds each agent’s context — preventing read-timeout / context-overflow
failures a single huge request would hit — and runs shards in parallel.
“Shard,” not “chunk.” A shard is a non-overlapping page range handed to
one concurrent extraction agent — the term carries the distributed-systems
sense of partitioning for parallelism. It is intentionally distinct from
chunking, which elsewhere in IDP means overlapping text windows (RAG-style)
or sequential list sub-batches; shards do not overlap and run in parallel.
extraction:
context_buffer: 0.30# ONE knob: keep 30% of each model window free ON TOP of the measured prompt overhead (auto-sizes everything below)
agentic:
enabled: true
runtime: step_functions# DEFAULT for agentic: per-shard Lambdas defeat the 900s timeout + resume
shard_token_budget: 0# 0 = AUTO-size from the model's context window (minus context_buffer)
max_pages_per_shard: 5# page ceiling per shard (timeout-critical; fixed default, not model-derived)
Model-aware auto-sizing (default).shard_token_budget: 0 means the per-shard OCR-token budget is derived from the extraction model’s context window minus context_buffer, minus an output reserve and an image reserve, minus the estimated prompt overhead for the class (system prompt, the task prompt with the schema rendered in, few-shot text, the tool schema and the Advanced-path restatement; the other agent tool specs are not counted; #775) — a 1M-context model (:1m) shards much larger than a 200K one, automatically, and a prompt-heavy class shards smaller instead of failing at the window. The confidence list-batch size is also derived, but from the confidence model’s output cap, the row’s column count and the geometry mode — not from context_buffer, which governs only the input-window budgets. The derived sizes, including prompt_overhead_tokens, are logged and shown in the Processing Report. Non-zero values pin an explicit override.
max_pages_per_shard is the timeout lever. It stays a small fixed default (5) rather than model-derived: the 900s Lambda limit is about sequential agent turns per shard (wall-clock), not context tokens, so a roomy token budget must not collapse a large doc back into one giant shard. Fewer pages/shard ⇒ fewer turns ⇒ each shard Lambda finishes well under 900s.
Advanced defaults to the resumable runtime.runtime: step_functions is now the agentic default: each shard is its own Lambda iteration in a nested Step Functions Distributed Map, so a very large section is not bound by the single-Lambda 15-minute limit and Step Functions retries only the incomplete shards (completed shards are reused from S3). in_process (asyncio within one Lambda) remains available but is still bound by that one Lambda’s 900s.
Confidence and bounding-box grounding are sharded too. Each shard runs its confidence assessment and grounds its own rows’ bounding boxes against only its own pages — so both scale per-shard and run concurrently. The final merge only concatenates already-scored, already-grounded rows (plus a fast top-up for any rows the assessment LLM omitted); it does not re-assess or re-ground the whole section. This keeps the merge step fast even on very large tables (previously a single full-section grounding sweep over thousands of rows could approach the merge Lambda’s 900s limit).
Large-document guidance. The defaults above are tuned to work out-of-the-box on large documents. Validated at scale on 100- and 200-page single- and multi-table documents (exact row counts, no loss/duplication, no timeouts).
See Large-Document Guidance for choosing between
Simple + separate confidence and Advanced sharding.
description: "A billing document listing items/services, quantities, prices, payment terms, and transaction totals"
properties:
InvoiceNumber:
type: string
description: "The unique identifier for this invoice, typically labeled as 'Invoice #', 'Invoice Number', or similar"
InvoiceDate:
type: string
description: "The date when the invoice was issued, typically labeled as 'Date', 'Invoice Date', or similar"
DueDate:
type: string
description: "The date by which payment is due, typically labeled as 'Due Date', 'Payment Due', or similar"
The solution ships predefined attributes for common document types (Invoices,
Forms, Letters, Bank Statements, etc.). You can add or edit attributes through
the Web UI: Configuration → Extraction Attributes tab → select class → Add New
Attribute (name, display name, description, optional formatting hints).
By default all classes use extraction.model. Override on a per-class basis with
x-aws-idp-extraction-model — useful when certain document types benefit from a
different model. The override works with both Simple and Advanced modes; classes
without it continue to use the global model.
extraction:
model: us.amazon.nova-pro-v1:0# Default for most classes
By default every class uses the global extraction.system_prompt and
extraction.task_prompt. Override either (or both) per class with:
x-aws-idp-extraction-system-prompt — overrides the system prompt for that class.
x-aws-idp-extraction-task-prompt — overrides the task prompt for that class.
This is useful when individual classes were independently optimized (e.g. via
separate AutoTune runs). Because the pipeline classifies first and extracts
per-class, the extraction step always knows which class it is processing. Classes
without these extensions use the global prompts. The override task prompt
supports the same placeholders: {DOCUMENT_CLASS},
{ATTRIBUTE_NAMES_AND_DESCRIPTIONS}, {FEW_SHOT_EXAMPLES}, {DOCUMENT_TEXT},
{DOCUMENT_IMAGE}, <<CACHEPOINT>>.
extraction:
model: us.amazon.nova-pro-v1:0
system_prompt: "You are a document extraction assistant."
task_prompt: |
Extract fields from this {DOCUMENT_CLASS} document:
x-aws-idp-extraction-system-prompt: "You are an expert W2 tax form data extractor."
x-aws-idp-extraction-task-prompt: |
Extract the following attributes from this {DOCUMENT_CLASS} form:
{ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
Document text: {DOCUMENT_TEXT}
type: object
properties:
employee_name:
type: string
description: "Employee name"
Note: These overrides also compose with
extraction.custom_prompt_lambda_arn. When a custom prompt Lambda is
configured, the per-class prompts are resolved first and passed to the Lambda
as defaults; any prompts the Lambda returns still take final precedence.
enabled: true# Advanced mode recommended for production
model: anthropic.claude-sonnet-4-5-20250929-v1:0
temperature: 0.0# Keep low for consistency
top_p: 0.1
top_k: 5
reasoning_effort: low# reasoning-capable models only; output is always model-max
system_prompt: |
You are an expert in extracting structured information from documents.
Focus on accuracy in identifying key fields based on their descriptions.
For each field, look for both the field label and the associated value.
When a field is not present, indicate this explicitly rather than guessing.
task_prompt: |
Extract the following fields from this {DOCUMENT_CLASS} document:
{ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
<few_shot_examples>
{FEW_SHOT_EXAMPLES}
</few_shot_examples>
<<CACHEPOINT>>
Here is the document to analyze:
{DOCUMENT_TEXT}
Format your response as valid JSON:
{
"field_name": "extracted value",
...
}
How prompts apply in each mode. Both modes use the same system_prompt and
task_prompt configuration; they are applied differently under the hood:
Simple:system_prompt is the Bedrock system message; task_prompt is the user message with document content; the model responds with JSON/YAML text that is parsed. No validation or retry.
Advanced (agentic):system_prompt is passed via custom_instruction and appended to the agentic system prompt; task_prompt is sent as the user message (text/images as content blocks); the agent returns a validated Pydantic model with automatic retry and self-correction.
You do not need separate prompts for agentic extraction — the better you define
your classes and attributes, the more accurate agentic extraction becomes.
If a document class is marked with x-aws-idp-exclude-from-processing: true (see
Excluding Static Pages in the Classification docs),
ExtractionService.process_document_section short-circuits for any section
classified as that class: no prompt is built and no LLM call is made. A
small stub result.json is written so downstream stages behave exactly as they
would for a real section:
"message": "Section 1 classified as 'PassportApplicationInstructions' ..."
}
section.extraction_result_uri is set to the stub URI. The stub is produced by
idp_common.section_exclusion.build_skipped_stub_result and written by
idp_common.section_exclusion.write_skipped_stub. Confidence assessment also
short-circuits for excluded sections (no LLM call, no additional stub — the
extraction stub is authoritative). See the demo at
notebooks/usecase-specific-examples/ds11-passport-application/demo.ipynb.
Three things happen to an extraction result after the model returns it and before
it is stored. All are configurable under Configuration → Extraction in the Web
UI, so any of them can be turned off without a redeploy if it causes trouble.
Repairs type/format mismatches deterministically — no model call, no cost:
Input
Field type
Becomes
"$1,234.00"
number
1234.0
"1.234,56" (European)
number
1234.56
"12.5%"
number
12.5 (magnitude preserved, not divided by 100)
"03/15/2024"
string + format: date
"2024-03-15"
"March 15, 1980"
string + format: date
"1980-03-15"
"Yes"
boolean
true
'{"City": "Anytown"}' (a JSON string)
object
{"City": "Anytown"} — the one cross-type repair that loses nothing
Every change is recorded in the section’s metadata.coercion, so nothing is
silently rewritten and you can audit exactly what was changed and why.
What it refuses to do. Anything genuinely ambiguous is left untouched and
recorded as a refusal rather than guessed:
"01/02/2024" — January 2nd or February 1st? (ambiguous_date)
"03/15/24" — a 2-digit year cannot be assigned a century (1924 or 2024 matters
for a date of birth)
"2024-03-15T09:00:00Z" into a date field — dropping a time component is data loss
"1,234.56" into an integer field — rounding would discard data
Anything across a type family — never string→object, never a scalar wrapped in an array
If your corpus has a known day/month convention, date_order: MDY or DMY
resolves the all-numeric ambiguous case. It never overrides a value that is
already unambiguous (a 15 cannot be a month whatever you set).
⚠️ The refusal covers coercion, not the pipeline.ambiguous_date only
engages when the model hands back the raw string. If the model normalizes
the date itself, it picks a day/month order silently — and there is no
refusal, no ProcessingIssue, and no metadata.coercion entry at all. The
guess is indistinguishable from a correct reading.
Measured on a single-page invoice containing Shipment Date: 03/04/1985
extracted into a format: date field, ground truth 1985-04-03 (D/M/Y):
extraction model
coercion
value returned
refusal recorded?
us.anthropic.claude-sonnet-4-6
off
1985-03-04
—
us.anthropic.claude-sonnet-4-6
on
1985-03-04
none
us.amazon.nova-lite-v1:0
off
1985-03-04
—
us.amazon.nova-lite-v1:0
on
1985-03-04
none
Both models resolved the ambiguity to M/D/Y on their own and emitted ISO
directly, so coercion never saw a string and had nothing to refuse. Note that
date_order does not help here either — it also only applies to values
coercion actually processes.
What this means for you. The property that holds is “coercion never
guesses a day/month order”, not “the pipeline never does”. For date-of-birth,
effective-date or any field where a transposed day and month is a correctness
problem rather than a formatting one, do not treat an unflagged date as
verified.
Two mitigations, both free. Instruct the model in the class or field
description to return dates verbatim, which puts the value back on
coercion’s path and makes the refusal authoritative; and/or set date_order to
your corpus convention so that path resolves rather than refuses. Note the
order matters — date_order alone does nothing here, because a
pre-normalized ISO value never reaches the code date_order governs. If your
corpus mixes D/M/Y and M/D/Y sources, neither mitigation is sufficient and
numeric dates need review.
This is a documented limitation, not planned work — see
#717
for the measurement and the reasoning. Detecting it automatically was
prototyped and rejected: the check can only fire on evidence that the document
is ambiguous, which on a normal US or EU corpus is a large fraction of all
dates, so it produced warnings faster than anyone could act on them. It cannot
tell you which reading is right — only that one was chosen.
How much does coercion actually change? Measured on a live stack: modern
models (Claude Sonnet 4.6, Nova Lite) already return correctly-typed values for
scalar fields, so coercion often fires zero times and changes nothing. It
fires substantially on long repetitive list rows (81 coercions across
100-row transaction lists in one benchmark), where model output drifts. Treat it
as a safety net for messy output and non-format-tolerant consumers — Athena
column typing, rule validation, API clients — rather than as an accuracy
improver. It did not move evaluation accuracy in either A/B we ran.
Bedrock rejects a single image over 5 MiB and measures the base64-encoded
payload, so a stored page image over 3.75 MiB — reachable from the shipped
defaults, which preserve original resolution — used to fail the whole document with
ValidationException: image exceeds 5 MB maximum at the extraction step
(#778). Such a page is now downscaled proportionally to fit (same format
first, JPEG only as a fallback), the reduction is logged, and the section records it
per page under metadata.image_downscale (original and final bytes, size and format,
passes, reason). A page that already fits leaves no entry. Set the service’s
image.target_width / image.target_height to keep pages inside the budget if you
would rather control the resolution than have it reduced per request; see
Image Processing Configuration.
The same mechanism also applies Bedrock’s many-image dimension cap: a request
carrying more than 20 image blocks caps every image at 2,000 px per side, so a
section over 20 pages has its pages downscaled to that before the request goes out
(#994).
Each reduced page appears in metadata.image_downscale with a reason naming the
pixel limit rather than the byte limit.
Simple extraction clamps at 21 pages — the whole section goes out as one request.
Advanced (agentic) extraction is judged per request, not per section, so its
threshold depends on how the pages shard. The agent re-sends its attached pages on
every turn and its view_image tool can add a further copy of a page to the same
request, so the figure one agent invocation will carry is doubled. On the shipped
defaults (max_concurrent_batches: 10, max_pages_per_shard: 5) each request
carries few enough pages that the clamp engages only at about 101 pages; at
max_concurrent_batches: 5 it is about 51, at 2 about 21, and with sharding off
(max_concurrent_batches: 1) it is 11. Because max_concurrent_batches caps the
number of shards as well as the parallelism, raising it makes each request smaller
and the threshold higher. Holistic classification, which sends a whole packet in one
request, is affected on the same >20-image rule.
Whether the clamp costs accuracy depends on the model — on Claude 4.7+, Opus 5,
Opus 5.5 and Sonnet 5 it is about 20% below the resolution they would otherwise use,
on older
Claude models it costs nothing, and it has not been measured against extraction
accuracy either way. See
A request with more than 20 page images
for the full trade-off and how to avoid the clamp.
If Bedrock does reject a request over its images, extraction fails with
ExtractionImageRejected and a message naming the image count and the largest
dimension, with the remedy that applies (extraction.image.target_width /
target_height) rather than shard-budget advice that cannot address it — and it
fails on the first attempt instead of retrying a request that will be rejected
identically every time.
Validates the result against the full class JSON Schema — most importantly the
format keywords (date, email, uri, uuid) that type validation alone does
not enforce. Runs on both Simple and Advanced extraction.
fail_action decides what happens when validation fails:
fail_action
Behaviour
Extra inference?
warn (default)
Records the outcome and raises an extraction_validation_failedwarning on the section; the data is kept
No — free
reject
Same, but the issue is an error and the result is recorded as not parsed, so the section’s Processing Report reads FAILED
No — free
escalate
Re-extracts only the failing fields with escalation_model, merged back over the fields that already validated
Yes
Validation is on by default precisely because the default action is free: it
turns an otherwise-silent schema violation into something visible at no cost —
the issue reaches the document list’s Processing Issues column and the
Processing Report tab, naming the failing fields and the first few concrete
violations, so you do not have to open the section result JSON.
escalate is the opt-in that spends money. Only the failing fields are
re-extracted and only those are merged back, so an over-eager escalation cannot
overwrite fields that already validated; if escalation fails, the original
extraction is kept unchanged.
⚠️ All three actions are visibility settings — none of them rejects a section.reject records parsing_succeeded: false, which is read by the section’s
Processing Report and the UI’s Processing Report tab and by nothing in the
status path, so the extracted values are still stored and the document still
reports COMPLETED. The one setting in extraction that turns a detection into an
outcome is
extraction.row_shortfall_action,
which fails the Step Functions execution.
Moved in v0.7. This block was extraction.agentic.validation. Stored
configurations are migrated automatically on read — no action required.
Forced tool use (extraction.forced_tool) — experimental, off by default
Coercion and validation repair or report a bad result after the fact. Forced
tool use tries to make one shape of bad result impossible in the first place: the
class schema is sent to the model as a required tool whose input is your
schema, rather than described in prose in the prompt, and the API constrains the
reply to that shape.
extraction:
forced_tool:
enabled: false# experimental; measure before turning it on
fallback_to_prompt: true# a prose answer still gets parsed (recommended)
Applies to Simple extraction only — Advanced (agentic) extraction already
sends a tool schema. Configurable under Configuration → Extraction → Schema
Enforcement (experimental).
Why it is off by default. Forcing constrains the structure of the reply, not
the accuracy of the values in it. A model that would have emitted a stray key
may instead emit a worse value that fits the schema, so this is not a free
accuracy win and it is not yet proven on real corpora. Measure completeness and
field accuracy on your own documents before enabling it.
When it is skipped automatically. Not every route can carry a tool
configuration. Models reached through a custom Lambda hook and the GPT-5.x
(Responses API) route fall back to the prompt, and the reason is recorded — so a
before/after comparison can tell “forcing changed nothing” from “forcing never
ran”, which are very different results.
A fixed failure to know about. Before this fix, a group attribute whose name
contains a space (Account Holder Address — normal in human-authored classes,
including the shipped bank-statement-sample) came back from Claude Sonnet 5 as a
JSON string rather than an object, so every section carrying it failed
validation: the schema’s $defs entry kept its spaced name and Sonnet 5 did not
resolve the $ref pointer to it. Definition names are now made wire-safe like
property names, the pointers are rewritten, and a group the model still returns as
a JSON string is parsed while its field names are restored — so the stored result
carries the names you authored, not the wire spellings. The name map follows each
$ref to its definition, so this holds for a class with several groups (the shipped
lending package has three) and never restores one group’s names against another’s.
A $ref to a shared string definition (an enum factored into $defs) is treated as
a string, so JSON-looking text in it is left alone. If you measured forcing before and
saw every section invalid, that was this (#783).
What gets recorded. Each section’s metadata.forced_tool holds requested,
honored (the model can accept a tool configuration and still answer in prose),
renamed_properties, renamed_definitions, and skipped with a reason where
applicable. honored is
the number to look at first: forcing that is not honored has not been tested.
Visible in the Prompt Preview. With Extraction mode Simple and this
setting on, Configuration → Prompt Preview gains a Tool Schema tab showing
the exact toolSpec — tool name, tool description, and input schema — and its
tokens are added to the previewed total, so the cost of turning enforcement on is
no longer invisible. The tab shows property names as you authored them and
says so, because the wire-safe rewriting below is reversed in the stored result;
the token estimate is taken from the compact form actually serialized onto the
request, not from the indentation added for readability.
Field names with spaces are handled. Bedrock restricts top-level tool
property names to ^[a-zA-Z0-9_.-]{1,64}$, which several shipped configuration
presets violate ("Invoice Number"). Such names are rewritten to a wire-safe
form for the request and restored to exactly what you authored in the stored
result — you never see the sanitized names, and no configuration change is
needed.
fallback_to_prompt: false turns an unhonored force into a parse failure
instead of falling back. That loses data by design and exists only to measure how
often forcing is actually honored; leave it on in production.
Classification splits sections by document type. When a packet concatenates
several records of the same type with no separator, there is no type change to
split on, so they land in one section — and extraction, whose class schema
describes one document, may return only the first record.
Each section now reports an instance count. In the Sections panel it is shown
only when there is something to say — a badge beside the section’s class when the
count exceeds 1, hoverable for detail. A normal single-document section shows just
its class name; so does a section whose count was never determined (older
documents, or extraction that failed before producing a result). The raw value is
on the API as Section.InstanceCount either way.
If a class’s schema is already modelled as a packet of records (one top-level
array, one element per record), name that array so the count can be derived:
classes:
- $id: patient_packet
type: object
x-aws-idp-instance-array: records# each element is one document
properties:
records:
type: array
items:
type: object
properties:
patient_name: { type: string }
This changes nothing about extraction output — it only tells the pipeline which
existing array is the instance axis — so it is safe to add to a working config.
If the model returns a JSON array for a single-document schema, every record is
now preserved (the first becomes the section result, the rest are recorded in
metadata.recovered_instances) and the section is flagged for review rather than
failed.
The hard case is the model returning a single object for a several-document
section. Only that one record exists in the response, so nothing can recover the
others — and until recently nothing reported it either: the section was SUCCESS,
the document COMPLETED, ProcessingIssueCount was 0, and the instance count
was 1. One record out of three, silently.
Extraction now asks the model, in the same inference, how many separate
documents of the class the supplied pages contain. When that answer exceeds the
number of records in the result, the section raises
extraction_multi_instance_suspected (severity warning) naming both numbers,
reports the model’s count as the section’s instance count, and emits a
MultiInstanceSectionsSuspected CloudWatch metric. The badge in the Sections
panel says the records are missing, not that they were extracted.
extraction:
multi_instance_detection:
enabled: true# OFF by default — turn it on for multi-record corpora
question: ""# blank = the shipped wording; supports {DOCUMENT_CLASS}
The question is editable, like every other prompt in the system — it is sent as the
description of the auxiliary property. Two clauses are load-bearing and should
survive any edit: “do not count pages, sections or repeated headers” (without it,
a document carrying an identical banner on each of four pages reads as four
documents) and “DIAGNOSTIC METADATA, not extracted document data” (so the model
does not treat the field as something to extract from the page).
Detection only. It never changes the extracted data, never fails a section,
and never turns on a schema flag for you. Fixing the loss is
x-aws-idp-multi-instance (below) or splitting the section.
OFF by default — but it is very good at its job. Measured on two real
labeled corpora, 80 paired Test Studio runs (same documents both sides, only the
toggle differing):
on 40 real bank-check images, against committed ground truth
multi-check images found
18 of 18
false alarms on the 22 single-check images
0
precision / recall
1.000 / 1.000
count reported exactly right
18 of 18 (2 to 8 checks)
token cost
input +1.8%, output −0.5%
It found every multi-check sheet and counted it correctly. But be precise about
what those 18 flags mean — they are a configuration finding on this corpus, not
averted data loss. The extracted rows were counted afterwards: 0 checks missing
in either arm. BANK_CHECK’s schema is a single checks array, so the class
already models several checks per sheet and nothing was collapsing. What detection
correctly spotted is that the class never declares its instance axis, so the
section reports 1 instance for a sheet holding 6 documents — fixed for free with
x-aws-idp-instance-array: checks, no schema change and no baseline migration.
The distinction matters, because “probe says 6, section says 1” cannot by itself
tell you whether records were lost: instance_count is 1 for any class with no
declared instance axis, whether the records are missing or sitting inside a
declared array. When detection fires, check the extracted data before concluding
anything was dropped.
Why it is off anyway. On a corpus with no multi-record documents to find, it
is pure cost: RealKIE-FCC-Verified lost about 1.3 accuracy points with it on
(0.7678 → 0.7552; worse on 14 of 40 documents, better on 1, sign test
p = 0.001), spread diffusely over four attributes rather than any single
failure mode. A default has to be safe for the deployment that gets no benefit,
so the default is off.
So: turn it on when a section can hold several records of one class and the
class schema describes only one. That combination is what loses records; either
half alone does not. Leave it off otherwise. It is per configuration profile, so a
multi-record corpus can have it while the rest of your deployment does not.
Two uses, and only one is a setting. As a diagnostic, turn it on once, act
on what it names, turn it off — that is how the BANK_CHECK misconfiguration
above was found. As a standing guard, leave it on where a corpus genuinely
keeps producing merged same-class sections and you want a warning on each one.
You keep authoring the class as one record — the wrapper is synthesized, so
you do not degrade a single-record schema by hand.
In the web UI both modes are one control, under Configuration → Document
Schema → the class:
Documents per section
( ) One document — the normal case; nothing changes
( ) Several — this class already lists them → Record array:records ▾
( ) Several — wrap my single-record class → instances[ ] → <Class> → { … }
The two modes are alternatives, so they are one question with three answers rather
than two separate toggles. The record-array picker appears only in the middle
branch, and the shape preview and the baseline-migration warning appear only in the
last one.
Opt-in per class, never automatic. Auto-detecting the shape would make every
single-document section pay for an extra nesting level and would move the
detection problem one stage later. Nothing changes for a class that does not set
the flag.
Two things that used to be gaps are now closed:
Discovery suggests it — and never sets it. When the sample Discovery
analyzes appears to hold several records of the discovered class, the model is
asked for a diagnostic count (the same question as the #753 probe, “count
documents, not pages”) and the job carries a suggestion: the job details page
shows “This sample appears to contain N ‘X’ records” with a one-click
Enable several documents per section action, the alternative (section
splitting) named alongside so the ambiguity is your call. The class setting is
never changed silently, because the shape change invalidates baselines
(GitHub #765). See Discovery.
Re-running Discovery on a class keeps the flag — along with every other
class-level x-aws-idp-* setting; the merge carries authored settings forward
(GitHub #764), and the suggestion above says so when the flag is already on.
The two keys are mutually exclusive (config validation rejects both on one class)
and answer opposite questions:
Your class describes…
Key
Schema change
Downstream impact
one record (the normal case)
x-aws-idp-multi-instance: true
wrapper synthesized
shape of inference_result changes — migrate baselines
a packet that already holds an array of records
x-aws-idp-instance-array: <property>
none
none
If you already hand-authored a List of DocTypeX class, stay on
x-aws-idp-instance-array — it costs nothing and changes nothing. Migrating to
Synthesize mode is optional; the payoff is correctly-keyed per-instance
confidence, Hungarian record matching in evaluation, and a per-attribute (rather
than one-giant-attribute) evaluation report.
An internal list is not a reason to avoid this. An invoice class with
line_items[] is a single-instance document with an internal list.
x-aws-idp-multi-instance: true on it is correct and gives
instances[i].line_items[j] — three invoices in one section. Configuration
validation only warns when a class’s top level is nothing but one record
array, because that is the case that would double-wrap.
This is the one part of the feature that can break a working deployment.
Evaluation compares a prediction against a stored baseline of the same shape.
A wrapped prediction against a flat baseline scores every field as
missing-on-one-side, so the class’s accuracy collapses to ~0 — measured live: a
correctly-migrated baseline scored 1.000 on the same document where a flat one
scored 0.000.
Evaluation now logs a warning naming the class and the exact migration command when
it sees the two shapes disagree, in either direction — so it is no longer silent.
But the warning is in the evaluation Lambda’s log, not on the report, and the score
is still 0: treat it as a safety net, not a substitute for migrating.
The script is idempotent (safe to re-run after an interruption), touches only
classes that set the flag, and refuses to flatten a multi-record baseline on
rollback because that would discard ground truth you authored.
It migrates shape, not content. Wrapping a one-record baseline gives
instances of length 1. If the document really contains three records — which is
the reason you turned the flag on — the other two were never in the baseline,
because the old pipeline could not extract them. You have to add them, or
evaluation will score the newly-found records as false positives. The script
lists every document it touched so that work is visible.
keyed instances[i].Field; each record scored independently
HITL review
alert labels become instances[i].Field — correct, but review task labels change
Reporting / Athena
one row per document, not per section, with a new record_index column. Existing column names are unchanged, so existing queries keep working — but (document_id, section_id) is no longer unique
Analytics agent
told about record_index for flagged classes
Z3 rules
address a record explicitly: …inference_result.instances[0].NetPay. A rule whose path no longer resolves logs what to write instead — a miss otherwise reads as “optional parameter absent” and the rule quietly stops firing
Public SDK
fields is unchanged (the raw shape), plus a new instances list; confidence now walks lists and groups
BDA mode
not applicable — this is a pipeline-mode (use_bda: false) feature
Advanced (agentic) extraction
the wrapper applies; the #753 detection probe runs on the single-agent path (a section short enough not to be sharded) and not yet on sharded sections, where per-shard counts have no sound reconciliation (#772)
What DEGRADES with the flag on — read this before turning it on
The wrapper moves every one of your properties one level down, and three checks
walk only the top level of a class schema. None of them is a correctness bug,
but all three quietly stop doing anything for a flagged class:
Check
What stops applying
BLANK vs MISSING field handling (x-aws-idp-source-page-types)
Nothing at the top level carries it any more, so the distinction is not applied at all for a flagged class.
”a declared list came back empty” (extraction_incomplete)
The only top-level array is now instances, so an inner list of a record coming back empty no longer raises it — and that is the largest silent-data-loss shape this pipeline has.
Confidence-prompt property descriptions
The prompt builder descends one level under an array, so a nested group or list inside a record loses its sub-field descriptions.
Evaluation report granularity
The per-attribute breakdown becomes one attribute (instances) carrying every field’s rows, instead of one per field. Every row is still there and still drillable — only the grouping is coarser.
Two more things to know:
Few-shot examples are not rewritten.x-aws-idp-examples prompts are
hand-authored text. If yours show a flat record they now contradict the requested
{"instances": […]} shape — and a flat answer is salvaged as exactly one
instance, so the loss looks like success. Re-author them wrapped. Configuration
validation warns when a flagged class carries examples.
The Prompt Preview shows the un-transformed schema, and says so — for both
the wrapper and the detection probe. The preview is
built in the browser from the class schema as stored, so for a flagged class the
prompt it renders is not the one the pipeline sends; it now carries a warning to
that effect rather than quietly misleading you. (It also omits the detection
probe when that is on.) The section’s stored metadata is the authoritative record
of what was sent. Duplicating the transform in TypeScript was considered and
rejected: two implementations that must stay in sync are a worse liability than
one documented divergence.
Confidence assessment produces a per-field confidence score (0.0–1.0) and an
explanatory reason for each extracted attribute, so you can gauge the reliability
of automated extractions and route uncertain fields to human review. In v0.6 it
is configured under extraction.confidence.
Per-attribute scoring: Individual confidence scores for each field, recursively for nested groups and list items. To keep responses compact, the default prompts ask the model to include a confidence_reasononly for lower-confidence leaves (below 0.9) — a high-confidence value emits just {"confidence": <score>}. This cuts assessment output tokens substantially (output tokens dominate assessment cost) without affecting the scores or thresholds. Widen or remove the threshold in the confidence task_prompt if you want a reason on every field.
Multimodal analysis: Optionally combines OCR text with document images.
Token-optimized: Uses condensed OCR text confidence data for 80–90% token reduction versus full OCR results.
Large-list batching: Long list fields (hundreds of rows) are automatically assessed in sequential batches (list_batch_size) so every row is scored — see below.
UI integration: Results appear in the web interface with color-coded confidence indicators and (with geometry) bounding-box overlays.
extraction.confidence.mode controls where per-field confidence and bounding
boxes are produced. For pros/cons and a recommendation, see
§0 Choosing a configuration; the table
below is the mechanical reference for where each combination runs:
confidence.mode
Extraction mode
Where confidence runs
Standalone Assessment step
separate (default)
Simple (non-agentic)
the standalone Assessment step
runs
separate
Advanced (agentic)
a second inference inside each extraction shard
bypassed (skip)
integrated
Simple (non-agentic)
the single extraction inference itself, in one pass
bypassed (skip)
integrated
Advanced (agentic)
within the extraction agent’s turn (see strategy note)
mode: separate# off | separate (default) | integrated
model: us.amazon.nova-lite-v1:0# default; far cheaper for the confidence pass
temperature: 0.0
top_k: 5
top_p: 0.1
reasoning_effort: low# only used if a reasoning-capable model is selected here
list_batch_size: 25# rows per assessment batch for large lists
system_prompt: |
You are an expert document analyst specializing in assessing the confidence
and accuracy of document extraction results.
task_prompt: |
# ... see prompt placeholders below
Behavior when disabled (enabled: false or mode: off): the assessment
Lambda is still invoked (minimal overhead) but returns immediately with logging
“Assessment is disabled via configuration” — no LLM calls, no S3 operations.
Defaults to enabled when the property is missing.
Migration note: The previous IsAssessmentEnabled CloudFormation parameter
has been removed in favor of this configuration-based control.
Confidence does not have to run as a separate downstream step:
separate on the agentic path runs a second inference inside each extraction shard (over that shard’s pages and extracted values) and collates on merge — per-field, page-ordered for list items, first-shard-wins for scalars — then grounds once in OCR geometry over the whole section. This reuses the sameAssessmentService.assess_results core and ground_assessment_geometry the standalone step uses, so the explainability_info output is identical; only the execution location differs. A single post-merge assessment would re-introduce exactly the context-window pressure sharding removes, hence per-shard.
integrated on the simple path is a true single inference: one Bedrock call returns values and confidence together. The prompt asks for a {"extraction": {...}, "confidence": {...}} envelope; the service splits that (values → inference_result, confidence → explainability_info after threshold-enrichment + OCR grounding), so the standalone Assessment step auto-skips (the single-inference cost win). For robustness the split also recognizes a field_assessment (or confidence) sibling key placed next to the extracted fields — a shape some models emit — and lifts it the same way (and strips it from inference_result so it never leaks). Only a genuinely flat response with no recognizable confidence falls back to the standalone step. Best for smaller documents where the whole doc fits one inference; for large docs prefer agentic or separate. Any list rows the single inference leaves unscored are retried (missing rows only, bounded) so large-list coverage still reaches 100%.
integrated on the agentic path produces confidence within the agent’s turn. Because extraction is delivered through a tool call, a hidden experimental setting extraction.agentic.integrated_confidence_strategy controls how confidence is produced. It is deliberately not exposed in the config UI; set it via idp-cli config-upload on a throwaway config version to A/B the trade-off. Both values yield identical explainability_info:
two_step(default): the agent extracts, then calls provide_field_assessment in a follow-up inference — a dedicated reflection pass over finalized values (often better-calibrated). ~3 inferences: extract → assess → close.
single_shot: the agent emits values and per-field confidence in one combined tool call, saving the middle inference. ~2 inferences. Multi-step (patched) extractions may still call provide_field_assessment once at the end so every row is assessed (unassessed rows padded with neutral confidence: null).
Both reuse the same prompt cache, so the saving from single_shot is one fewer inference, not a change in cached-token economics.
Automatic bypass of the standalone step. When extraction has already written
explainability_info to the section result, the Assessment Lambda’s intelligent
skip detects it and returns immediately without a second LLM call — no duplicate
cost and no state-machine change is required. The document status stays
EXTRACTING while in-shard assessment runs.
For documents with large lists (bank statements with hundreds of transactions,
line-item tables, brokerage holdings), a single confidence inference over the
whole section under-enumerates or omits the list, leaving most rows unassessed.
Every confidence path batches large lists automatically — the standalone
Assessment step (Simple + separate), the in-shard pass (Advanced + separate),
and the inline-confidence retry (integrated) all share one implementation:
It slices the largest list field into extraction.confidence.list_batch_size
chunks (default 25).
Each chunk is assessed sequentially, sharing the scalars/context.
The concatenated per-row assessments are reconciled so every list cell
receives its own confidence and (in an OCR-backed geometry mode) its own
bounding box.
So large lists are handled with no extra configuration regardless of the
Simple/Advanced or separate/integrated choice. The one knob is list_batch_size,
and it is a ceiling: the size actually used is derived per list from the
confidence model’s output cap, the row’s column count and the geometry mode, and is
only ever smaller. Lower it to force smaller batches than the derivation;
raising it above the derived size has no effect.
extraction:
confidence:
enabled: true
mode: separate# separate (default) | integrated | off
list_batch_size: 25# rows per assessment batch for large lists
Automatic recovery when the model truncates (self-healing).list_batch_size
is a row count, but the model’s real limit is its max output tokens. When
per-row output is large — most notably with geometry.mode: llm/llm_grounded
(a bounding box per cell, ~3× the output) — a batch can overflow a small-cap
model’s ceiling (e.g. Amazon Nova Lite caps at 10,000 output tokens). A
truncated response is unparseable, which used to silently assign a default
0.5 to every field and leave list rows unscored. Advanced mode now heals this
automatically, cheapest-first:
Token-aware first-pass sizing. The first batch is sized from three inputs:
the confidence model’s output cap, the column count of the list’s widest
row, and whether the geometry mode adds a bounding box per cell. On a
10,000-token-cap model (Nova Pro) with llm_grounded that is 13 rows for a
3-column list and 5 for 8 columns; without bounding boxes three times as many
fit. This only ever
shrinkslist_batch_size — it never grows past your configured ceiling, so
raising the ceiling above the derived size has no effect. Where a model family
has a measured loop ceiling it applies too: Amazon Nova Lite/Micro score at
most 12 rows per call, because at temperature 0 a 25-row batch made Nova Lite
repeat the same row object until it hit its 10,000-token cap on every run (~60 s
and 10,000 output tokens per document), while 13 rows looped occasionally and 8
never. On those two models each call also requests only the output budget
a correct answer needs (about 40 tokens per scalar or list cell, three times
that with LLM bounding boxes, plus overhead; floor 2,000, cap the model’s
maximum) rather than the model’s full cap, so a degenerate response is cut off
early and recovered by the steps below instead of running to the cap. Every
other confidence model keeps requesting its maximum output.
Recursive splitting. Any batch that still truncates is halved and
re-assessed until it fits.
Model escalation. If rows are still unscored after shrinking + retries,
the remaining rows are re-assessed on a stronger confidence model (bigger
output cap). ON by default; configure with:
Per-class override: x-aws-idp-confidence-escalation-model. Set
escalation_model: null to skip the model step.
Give up when the batch is already one row. Halving only converges while a
smaller batch can fit. If the model still truncates with a single row in
the call, that one row’s confidence output exceeds the model’s cap and no batch
size can work, so the ladder stops there instead of retrying: the futile
same-model retry rung is skipped, at most one escalation round runs (a bigger
output cap is the only remedy that can legitimately succeed), and the section
reports assessment_row_too_large (error) naming the model, its output cap,
the field and class, and the offending row’s approximate serialized size.
Before this guard there was no terminal condition for that case: the impossible
call was re-run through the retry rounds and the bisection tree, and the section
ended up reporting the generic assessment_incomplete, whose remedy (a smaller
batch) cannot work. Measured on a shape that batches — 8 rows at batch 4 — the
guard halves the primary model calls (28 → 14); on the single-outer-row shape
that #894
reports, the call count is unchanged and what improves is the diagnosis.
The known trigger is a multi-instance class
(x-aws-idp-multi-instance: true): the wrapper makes the instance array the
outer list, so one “row” is a whole document instance carrying its own long
inner list (e.g. a 100-row Transactions table). ⚠️ Such a class still
cannot be fully assessed. The batch sizer measures only the outer row’s
column count (it reports cols=2 per_row~80 for a row holding 100
transactions) and derives a batch size that is wrong by orders of magnitude;
fixing the sizer to descend into inner lists is tracked as open issue #894.
What this guard changes is the reporting, and the wasted calls on shapes that
batch — not the outcome for such a class. ⚠️ The symptom originally reported on
#894 (an Assessment Lambda hitting its 900 s limit three times,
Sandbox.Timedout ×3, leaving the document in ASSESSING after a 45-second
extraction) is not explained by this loop, so treat it as still open: the
same-model retry rung already stopped on no progress before this change, and the
ladder’s wall-clock deadline guard was already in place in the release where
those timeouts were observed. The single gap in that guard has since been closed
(#958): every recovery call — retry and escalation alike — now checks the
Lambda’s remaining time before it is made, as further bisections already did, so
a run that would have spent its whole budget on partially-successful retry
rounds stops at the last call that fits and keeps every row it recovered. With
rows still unscored the section reports assessment_incomplete naming the time
budget (the assessment_deadline_reached warning is for a run that was cut
short and scored every row regardless). That was the most concrete lead on the
timeout, but it is not confirmed as its cause. Until #894 is fixed, score such a class
with a large-output-cap confidence model (escalation_model), or split the inner
list into its own class.
This activity is recorded in the section’s
metadata.assessment_batch_split_stats (derived_batch_size,
escalation_model, rows_recovered_by_escalation, unrecoverable_rows,
oversized_row_*, …) and
(for agentic) an ⚠ Assessment Batch Splitting block in the processing report.
If rows remain unscored even after escalation, the durable fix is to reduce
per-row output — e.g. set geometry.mode: ocr_only (the default) so boxes come
from OCR value-matching rather than the model.
Surfaced in the UI. Whatever the self-healing ladder does (or can’t do) is
recorded as a structured processing issue on the section — severity
(error / warning / info), code (e.g. assessment_incomplete,
assessment_row_too_large, assessment_coverage_incomplete,
assessment_recovered_with_retries, assessment_deadline_reached), a
user-facing message, and a technical root_cause. These are persisted to
DynamoDB and shown in the Web UI: a Status column on the document’s Sections
panel (hover for the message + root cause), a Processing Issues count column
on the document list, and a structured list at the top of the section
Processing Report tab. A document that quietly self-healed — or one where a
row genuinely couldn’t be scored — is therefore visible at a glance.
A failed confidence pass no longer fails the document. Confidence scoring
runs after extraction has written its results and been paid for, so a
deterministic confidence failure that discards them destroys the expensive part
to report the loss of the advisory part — two observed runs threw away $17.34 and
$7.05 of extraction that had scored 1,200 of 1,200 rows at 1.000 cell accuracy,
because the confidence model rejected an oversized input with
ValidationException: Input is too long for requested model.
(#901).
A deterministic (non-retryable) failure in the Assessment step now degrades
instead: the extracted data is returned and the document succeeds, with the
missing confidence recorded as an error-severity
assessment_failed_confidence_unavailable processing issue on the section.
Transient failures (throttling, read timeouts, 5xx) are unchanged — they still
raise so Step Functions retries the section. ⚠️ Because confidence is absent on a
degraded section, HITL confidence routing and the UI threshold signals do not
apply to it; check the Status column of the Sections panel — that is where this
particular issue shows. It is written to the section record only, not into the
section’s result.json, so unlike issues from a successful assessment run it
does not appear in the Visual Editor’s Processing Report tab (which renders
metadata.processing_issues from that file). Re-batching the oversized
confidence input so the pass succeeds rather than degrading is still open as
part of #901.
Operators: this is also a CloudWatch signal. Because a degraded section no
longer fails its document, a systemic confidence failure — a confidence model
whose input limit every section exceeds, a revoked bedrock:InvokeModel grant, a
model id not enabled in the region — produces no failed executions and moves none
of the failure alarms. Each degrade therefore publishes
AssessmentConfidenceUnavailable to the stack’s own metric namespace. So does a
section that reached the Assessment step with nothing to assess — no
extraction result, no pages, an empty inference_result, or none of its pages
present in the document — which records
assessment_skipped_confidence_unavailable instead and is likewise invisible in
the document’s own status. The metric covers both, because the alarm’s question is
whether sections are coming back without confidence; the issue code says which
happened. Two exceptions, and you will meet them on ordinary documents: a section
whose class is in configuration with no attributes to extract, and a section
classification could not classify at all, publish nothing and record nothing
here, because extraction skipped the model deliberately for both. The second
covers every page labelled unclassified — a blank page, a page whose
classification failed — so do not expect a data point for those; the
classification stage
reports them, where the remedy is. A section whose named class is missing from
the configuration is not an exception and does publish: the section was
expected to hold data and holds none. How often that arrives depends on your
classification configuration — on the default it needs a configuration edit, but
textbasedHolisticClassification and enforceValidClasses: false both store an
out-of-vocabulary prediction verbatim, so there the rate follows model output and
ConfidenceUnavailableThreshold is the lever.
AssessmentConfidenceUnavailableAlarm fires at ten or more in fifteen
minutes — on volume, not on the first occurrence, since one such section is an
expected outcome (#996,
#1006).
See Monitoring for the metric,
the alarm and the causes worth checking first.
This replaces granular assessment. The former “granular assessment”
service (a separate thread-pool fan-out with DynamoDB caching) has been
retired and deleted. Large-list batching is its full replacement: complete
per-cell confidence and geometry at substantially lower Bedrock cost on a
120-row bank statement, with equal accuracy (granular actually produced 0%
geometry). No cost percentage is quoted here deliberately — see “Why there is no
percentage here” in the retirement note.
Any legacy granular.* keys still validate but are ignored — no config edit is
required. See Granular Assessment Retirement.
For very large or complex documents, prefer Advanced (agentic)
extraction — it shards both extraction and assessment and produced the
best-calibrated confidence in A/B testing. Simple + separate remains fully
viable for large lists via the batching above. See
Large-Document Guidance.
Geometry gives each extracted field a bounding box locating it in the
document, which the Web UI renders as an overlay linking form fields to their
place on the page. Geometry is an output of extraction, configured under
extraction.geometry. The UI-compatible geometry format matches BDA mode:
geometry is an array of { boundingBox: {top, left, width, height}, page }
with normalized 0–1 coordinates and 1-based page numbers, supporting nested
group attributes and list items recursively.
extraction.geometry.mode controls where field bounding boxes come from:
geometry.mode
Source of boxes
Notes
ocr_only(default)
Real OCR lines only
Model is not asked for boxes; cheapest and most accurate
llm_grounded
LLM estimates, grounded in OCR
Model emits boxes; service replaces them with real OCR boxes where the value matches
llm
LLM estimates as-is
Escape hatch; no grounding
off
None
No geometry produced
extraction:
geometry:
mode: ocr_only# ocr_only (default) | llm_grounded | llm | off
task_prompt_bbox: |# only used by llm_grounded / llm modes
# ... spatial-localization guidelines (see below)
v0.6 rename. The LLM-box modes were previously llm_with_ocr_grounding /
llm_only, and the legacy assessment.ground_geometry_in_ocr: false maps to
llm. Old configs are migrated on read.
ocr_only (default) — OCR grounding via pageData.json
In ocr_only mode the model is not asked for boxes at all. Each field’s
geometry is derived by matching the extracted value text against real OCR lines
in the consolidated pageData.json artifact (Amazon Textract, or the Mistral OCR
LambdaHook; the Cohere Parse hook contributes boxes for tables and figures only).
This is cheaper (no bbox tokens in the response) and more
accurate (OCR boxes beat LLM-estimated boxes, which models frequently
hallucinate). A field with no OCR match simply has no geometry (geometry is
advisory).
Repeated values are disambiguated by row order — when the same value appears
on multiple rows (e.g. a repeated amount in a table), the i-th assessed list item
maps to the i-th occurrence in reading order.
Format-aware matching. Extraction often canonicalizes a value to a schema
format that differs from how it appears in the document (a date extracted as
2022-04-04 but printed 04/04/2022; an amount 1234.00 printed $1,234.00; a
phone +15551234567 printed (555) 123-4567). Matching is bridged three ways:
Format variants — the value is re-rendered in common surface forms and each is matched.
Type-aware equality — value and OCR line are parsed as the same logical date/number/phone and compared (robust to any rendering).
Character-level Levenshtein — last resort for OCR noise (e.g. Acme vs Acrne).
The field’s logical type comes from its JSON-Schema type/format when present,
else inferred from the value. Format-bridged matches are tagged
geometry_source: "ocr-normalized" and Levenshtein near-misses "ocr-fuzzy",
distinct from exact "ocr" hits so they stay auditable.
- Make bounding boxes tight around the actual text content
</spatial-localization-guidelines>
Provide confidence assessments with spatial localization in JSON:
{
"attribute_name": {
"confidence": 0.85,
"confidence_reason": "Clear text with high OCR confidence",
"bbox": [100, 200, 300, 250],
"page": 1
}
}
Automatic coordinate conversion. When the LLM returns bbox data, the service
automatically detects bbox/page, converts from the 0–1000 normalized scale to
0–1 decimals, transforms [x1, y1, x2, y2] into {top, left, width, height},
and processes nested/list attributes recursively. Reversed coordinates are
auto-corrected; invalid or incomplete boxes are dropped (confidence assessment
continues).
llm_grounded: the LLM box is grounded in real OCR coordinates where the value matches, falling back to the LLM box otherwise. The LLM box also disambiguates repeated values by spatial proximity.
llm: the model’s boxes are used as-is with no grounding.
For ocr_only and llm_grounded, grounding is a post-LLM, server-side
enrichment step (it does not change the prompt or its token budget). After
extraction/assessment, the service reads the consolidated per-page
pageData.json directly from
S3 and matches each extracted value to an OCR line.
For each assessed leaf field, the service:
Loads pageData.json for the section’s pages via Page.ocr_page_data_uri (older documents without this artifact are skipped — see fallback).
Matches the value against the page’s OCR lines[], in precision order: exact line → value contained in a line → multi-line span (boxes unioned) → single-line fragment → token-overlap fuzzy (Jaccard ≥ 0.6).
Disambiguates repeated values spatially (for llm_grounded): all candidates are collected and the one whose box center is nearest the LLM-estimated box is chosen. Without a usable LLM reference box, an ambiguous match keeps the LLM box rather than risk attaching the wrong row’s geometry.
On a confident match, replaces the field’s geometry with the real OCR box and adds provenance keys.
Output additions on grounded fields:
{
"account_number": {
"confidence": 0.95,
"confidence_reason": "Clear text with high OCR confidence",
geometry_source — "ocr" (grounded in a real OCR line), "ocr-paragraph" (grounded in a paragraph-level box shared by sibling lines, as the Mistral hook produces), "ocr-normalized" / "ocr-fuzzy" (format-bridged / Levenshtein matches), or "llm" (kept the LLM-estimated box).
ocr_confidence — the matched OCR line’s confidence (0–1), when available. Informational only; the LLM confidence/confidence_reason are never overwritten, so HITL triggering and threshold alerts are unaffected.
Safe fallback (backward compatibility). Grounding degrades to exactly the
prior LLM-only behavior — the worst case is no change:
No pageData.json (older documents, or a missing URI) → keep the LLM box.
OCR backend provides no geometry (plain Bedrock LLM OCR, Chandra hook, none backend; geometryAvailable: false) → keep the LLM box.
Value doesn’t confidently match any OCR line, or repeated values can’t be disambiguated → keep the LLM box.
Because grounding reads pageData.json from S3 (not the prompt), it adds zero
tokens to the request.
Fields whose confidence falls below a threshold can be routed to human review.
HITL is configured under the top-level hitl block in v0.6:
hitl:
enabled: true
confidence_threshold: 0.85# fields below this are flagged for review
Human review integrates confidence scores and geometry so reviewers see exactly
which fields need attention and where they appear on the page. For the full
workflow, the review UI, and Amazon A2I integration, see
Human Review.
Confidence thresholds identify extraction results that may require review.
Thresholds can be set globally or per-attribute, and the UI provides immediate
color-coded feedback. The system automatically adds confidence_threshold values
to the explainability_info based on configuration.
Global threshold — system-wide requirement for all attributes:
{
"explainability_info": [
{
"global_confidence_threshold": 0.85,
"YTDNetPay": { "confidence": 0.92, "confidence_reason": "Clear match found in document" },
For type: array attributes, each item sub-field carries its own threshold,
resolved from the item schema. This works whether the item schema is inline
under items.properties or referenced with $ref into $defs:
Each row of w2_copies is scored per column: w2_box_a_employee_ssn against
0.8, w2_box_1_wages against 0.9, and w2_form_year — which declares no
threshold — against hitl.confidence_threshold.
Resolution order for any field is:
The field’s own x-aws-idp-confidence-threshold (for list columns, the one
inside the resolved items / $defs schema).
The x-aws-idp-confidence-threshold on the array attribute itself, if set.
hitl.confidence_threshold.
This applies to both processing modes. In BDA mode, per-field thresholds are
resolved for the blueprint-driven custom_output (the section results that match
your class schema). Page-level standard_output confidence is generic document
analysis rather than named schema fields, so it uses hitl.confidence_threshold.
Note: if hitl.confidence_threshold is 0.0, any field without an explicit
x-aws-idp-confidence-threshold is effectively never flagged. Set a meaningful
default (e.g. 0.8) so unannotated fields still get reviewed.
Note — nesting depth. Two resolvers are involved, and they differ below the
first array level. The path-based resolver (resolve_threshold_for_path, used by
the BDA HITL alert path) walks arbitrary nesting — objects inside array items,
arrays inside groups, nested arrays. The enrichment function
(enrich_assessment_with_thresholds, used by the pipeline assessment and BDA
result.json paths) resolves only one level of array item sub-fields, so a
threshold declared on a field below an array item (e.g.
w2_copies[].address.zip) falls back to hitl.confidence_threshold there while
the HITL alert path honors it. For the common case — flat array items with
per-field thresholds, which is what $defs item schemas normally describe —
both paths agree exactly. This is pinned by
test_resolvers_diverge_on_nesting_below_array_items, so if you need the deeper
case, unify the two paths and update that test rather than working around it.
⚫ Black: Confidence available but no threshold for comparison
Interface coverage includes the Visual Editor tab (split-pane document image
form editing, bounding-box overlays, recursive nested display, inline editing
with change tracking), the JSON Editor tab, the Edit history tab
(audit trail with field-level diffs), smart filters (low-confidence,
evaluation mismatches, collapsible tree), and nested-data support:
Threshold best practices. Set higher thresholds (0.90+) for critical
financial or personal data; use per-attribute thresholds for different data
types; establish reasonable global defaults (0.75–0.85); start conservative and
tune based on accuracy analysis. Route below-threshold extractions to
Human Review.
Both extraction and confidence prompts support precise control over where
document images appear, using the {DOCUMENT_IMAGE} placeholder.
Without the placeholder: images are automatically appended after the text content.
With the placeholder: images are inserted exactly where {DOCUMENT_IMAGE} appears.
extraction:
task_prompt: |
Extract the following fields from this {DOCUMENT_CLASS} document:
{ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
Examine this document image:
{DOCUMENT_IMAGE}
Text content:
{DOCUMENT_TEXT}
Respond with valid JSON containing the extracted values.
Common patterns: visual-first (image before instructions), verification
(image after text to correct OCR errors), and mixed content (image shows
tables/stamps/signatures the text misses). The placeholder works seamlessly with
few-shot examples. For confidence prompts, images are only processed when
{DOCUMENT_IMAGE} is explicitly present (text-only assessment otherwise) — and
exactly one occurrence is required when present. The prompt is the only
thing that decides this: geometry.mode does not affect whether page images are
attached, because a confidence pass that is never asked for bounding boxes still
needs the image to judge visually-evidenced fields (signature / checkbox / stamp
booleans, handwriting, struck-through values). To make the confidence pass
text-only — e.g. to cut input tokens on a long table — remove {DOCUMENT_IMAGE}
from the confidence task_prompt.
Multi-page handling. Images are processed in page order with no image count
limits (following the Bedrock API removal of image count restrictions); documents
of any length are processed without truncation.
Benefits: enhanced accuracy, better table/form handling, improved handwritten
content and visual-only elements (stamps, logos, checkboxes), and OCR-error
verification.
Both extraction and confidence support configurable image dimensions under an
image: sub-block. Empty strings or unspecified dimensions preserve the
original document resolution for maximum accuracy (this is the default and
recommended for critical extraction):
extraction:
image:
target_width: ""# Empty string = no resizing (recommended)
target_height: ""# Empty string = no resizing (recommended)
confidence:
image:
target_width: ""
target_height: ""
Configure specific dimensions when optimizing for performance:
Migration from previous versions. The old behavior defaulted empty strings
to 951x1268 resizing; the current behavior preserves original resolution. To
keep the old behavior, set target_width: "951" / target_height: "1268"
explicitly.
CachePoint caches partial computations to improve performance and reduce costs.
Enable it by placing <<CACHEPOINT>> tags in prompt templates to mark where the
model should cache preceding prompt components:
extraction:
model: us.amazon.nova-pro-v1:0# Must be a CachePoint-compatible model
task_prompt: |
<background>
You are an expert in business document analysis and information extraction.
</background>
<<CACHEPOINT>> # Cache the instruction portion
Here is the document to analyze:
{DOCUMENT_TEXT}
Supported models include:
us.anthropic.claude-haiku-4-5-20251001-v1:0
us.anthropic.claude-haiku-5-5
us.anthropic.claude-sonnet-5 (and the other Claude 4.x/5 Sonnet/Opus IDs)
Optimal placement: separate static content (system instructions,
few-shot examples — cacheable) from dynamic content (document text — not
cacheable), placing the tag right before the document text. Cached input tokens
are billed at a reduced cacheReadInputTokens rate (roughly 10× cheaper than
standard input tokens).
The extraction service auto-detects and parses both JSON and YAML LLM responses
via extract_structured_data_from_text(). YAML is more token-efficient (~10–30%
fewer tokens; ~25% typical): no quotes around keys, compact nested syntax,
natural multiline support. All existing JSON prompts continue to work unchanged —
no configuration changes required, with intelligent fallback between formats if
parsing fails.
# JSON response (traditional)
extraction:
system_prompt: "You are a document assistant. Respond only with JSON. Never make up data."
task_prompt: |
Extract the following fields from this {DOCUMENT_CLASS} document and return a JSON object:
{ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
Document text: {DOCUMENT_TEXT}
JSON response:
# YAML response (more token-efficient)
extraction:
system_prompt: "You are a document assistant. Respond only with YAML. Never make up data."
task_prompt: |
Extract the following fields from this {DOCUMENT_CLASS} document and return YAML:
Improve accuracy by providing examples within each document class configuration
(class-specific — only examples from the same class being processed are
included). Use the {FEW_SHOT_EXAMPLES} placeholder in the task prompt:
Function name must start with GENAIIDP- (required for IAM permissions).
Must return valid JSON with system_prompt and task_prompt_content fields.
Available in Pipeline mode (Patterns 2 and 3 historically).
Input payload provides config, prompt_placeholders (DOCUMENT_TEXT,
DOCUMENT_CLASS, ATTRIBUTE_NAMES_AND_DESCRIPTIONS, DOCUMENT_IMAGE as S3 URIs),
default_task_prompt_content, and serialized_document. Required output:
{
"system_prompt": "Your custom system prompt based on document analysis",
"task_prompt_content": [
{ "text": "Your custom task prompt with business logic applied" },
{ "image_uri": "<preserved_placeholder>" },
{ "cachePoint": true }
]
}
Error handling is fail-fast: Lambda invocation failures, invalid response
format, function errors, and timeouts all cause extraction to fail with detailed
messages. Image bytes are never sent (S3 URIs only) to minimize payload size.
Only GENAIIDP-* functions can be invoked (scoped IAM), and all invocations are
logged. For complete examples see notebooks/examples/demo-lambda/README.md and
notebooks/examples/step3_extraction_with_custom_lambda.ipynb.
Advanced (agentic) extraction is generally the better fit. It shards both extraction and confidence assessment into token-/page-budgeted ranges, bounding each inference’s context and preventing read-timeout / context-overflow failures. It produced the best-calibrated confidence in A/B testing. For 100+ page documents prefer runtime: step_functions and raise max_concurrent_batches (e.g. 10). For large tables specifically, enable table parsing (§1) when OCR emits Markdown tables.
Simple + separate confidence still handles large lists via large-list batching (list_batch_size) — full per-cell confidence and geometry with no extra configuration. This is the direct replacement for the retired granular assessment.
Rule of thumb: a document that is large only because it contains a long list (e.g. a 300-row bank statement) is well served by Simple + separate + batching. A document with a very large single section (dense multi-page narrative or multiple large tables) is better served by Advanced sharding, which also splits the extraction work, not just the confidence pass.
Extraction can lose rows silently: measured on synthetic bank statements with
exact ground truth (Sonnet 5, one run per size, after the #726 over-splitting fix so
each statement is one section), simple mode returns complete lists (recall 1.000)
through 400 rows / 9 pages, then 43 of 800 rows at 17 pages while reporting
COMPLETED with no processing issue, and fails outright with Input is too long
for requested model from about 25 pages. (Earlier figures of “complete through
~1,600 rows” were an artefact of over-splitting, which cut large statements into 6–18
sections; see docs/benchmarking/config-guidance.md §3.) Two things make this
particularly easy to miss:
Scalar accuracy is unaffected. The document’s non-list fields extract
perfectly whether the table came back complete, partial, or not at all — so a
quality metric based on field accuracy looks healthy.
A truncated run is cheaper. Cost fell from $1.78 to $1.04 when a run
truncated, so cost monitoring will not flag it either.
So it must be detected structurally. Four signals are raised as processing
issues — reaching the document list’s Processing Issues column, the
Processing Report tab and the tracking table’s sparse
HasProcessingIssues attribute — on both Simple and Advanced modes:
Code
Severity
Fires when
extraction_incomplete
warning
A schema-declared list came back empty, null, or absent from the response entirely.
extraction_list_truncated
warning
A list returned fewer rows than its schema minItems — the one unambiguous truncation signal available without ground truth.
extraction_sparse
info
Fewer than min_population_ratio of the schema’s leaf fields were populated.
extraction_rows_below_ocr_estimate
warning (error under row_shortfall_action: fail, which is then the only one of these that changes the document’s status — see below)
A list of objects returned fewer than half the rows found in the section’s OCR tables of the same shape — tables whose column count equals the list item’s property count — and those tables hold at least 30 rows. This is the ground-truth-free signal for the Simple-mode case above (43 rows extracted from an 800-row statement), which passes every other check because the list is non-empty and the scalars are right. Only Markdown tables count (rows starting with a pipe under a `
A fifth issue is raised by schema validation
rather than the completeness checks:
Code
Severity
Fires when
extraction_validation_failed
warning (error under fail_action: reject)
The result still violates the class JSON Schema after extraction (and after escalation, if enabled).
Making a materially incomplete list fail the section — extraction.row_shortfall_action
A processing issue, at any severity, does not change a document’s status: it is
recorded on the section, counted, and rendered in the UI, and the document still
reports COMPLETED. That is true of every incompleteness signal in the
pipeline, error severity included, and it is why a 1,200-row statement carrying
43 rows reported success — and, because a truncated run is cheaper than a
complete one, why neither status nor cost flagged it.
extraction.row_shortfall_action is how a deployment makes the
extraction_rows_below_ocr_estimate detection decide the outcome:
Value
Outcome
warn (default)
Record the issue and report success.
fail
The rows that were extracted, the issue and the processing report are written to the section’s result.json first; the section then fails with ExtractionOutputIncomplete, which is deterministic and in no Retry list, so it fails once and in seconds. The document’s status is FAILED, the Step Functions cause is the sentence naming the rows extracted, the OCR row estimate and the remedy, and the section’s own record carries both that shortfall issue and extraction_failed — see below.
It does not change when the shortfall is detected — the floor of 30 matching
OCR rows and the “fewer than half” ratio are identical under both values, so the
sections that fail under fail are exactly the ones that warn under warn. And
it is not Simple-mode-only: the check and the outcome are shared by both modes,
because a truncated Advanced section tells the same lie. Advanced mode shards, so
it reaches this far less often.
A section whose extraction failed is recorded on the section itself, so the
document’s Sections panel shows which section failed and why — as Failed in the
Status column, distinct from the Incomplete a section that was flagged but
accepted shows. That applies to every raising extraction failure, not only a row
shortfall: ExtractionInputTooLarge, ExtractionImageRejected,
ModelInvalidToolUseSequence and ExtractionOutputIncomplete all leave an
error-severity extraction_failed issue whose root_cause is the exception’s own
explanation and remedy — the same sentence the Step Functions cause reports, which
is unchanged.
Two issues appear together under row_shortfall_action: fail, and they say
different things. extraction_rows_below_ocr_estimate names what was lost and how
to accept it; extraction_failed says the section did not succeed. The first
cannot carry that on its own, because it is written at warning severity under
warn — where the document completes — and at error under fail.
The rows that were extracted are still in the section’s result.json and the
section still points at it, so a partial result stays readable in the Visual
Editor rather than being discarded with the failure.
A transient failure is not marked. A throttle or a read timeout is retried by
the state machine, so flagging the section would show it failed for as long as that
ladder runs and then clear itself. Only a failure that will not be retried is
recorded. If a retry ladder exhausts every attempt, the document fails with the
explanation in the Step Functions cause and the section is not flagged.
A very long explanation is abridged in the middle. A processing issue’s details
are bounded (4 KB for the technical cause, 1 KB per value in its structured payload,
both in bytes) because they all share one DynamoDB record with every other issue on
the section, and some exceptions echo extracted document content — a schema-validation
failure on a 900-row list renders the whole list into its message, about 57,000
characters per failing field. What is removed is the middle, so the failure at the
start and the remedy at the end both survive. The unabridged text is in the section’s
result.json and in the CloudWatch log for the step.
One case is abridged from a place you might not expect. A classification issue puts
the section’s page list at the end of its technical cause, so on a section with
more than about 530 pages the abridgement falls inside that list rather than after it.
Nothing is lost: the same page ids are in the issue’s structured payload and in its
one-line message, both of which the Sections panel shows.
Why fail is opt-in, and what to check before turning it on
Not caution — a property of the evidence. Matching is on width only and
matched tables are summed over the whole section, so any table whose column
count happens to equal the list item’s property count is counted as evidence
about that list. For a 2- or 3-property array that models an entity group
rather than table rows, that evidence has no legitimate contribution at all, and
a group-shaped array is structurally indistinguishable from a row-shaped one.
The default preset shows it. Bank-Statement.account_summary has two properties
(summary_desc, summary_amount). A monthly statement carries a five-row
two-column Account Summary, a sixty-row five-column transaction table, and a
31-row two-column Daily Balance table. A fully correct extraction — all five
summary rows, all sixty transactions — scores 5 / 38 = 0.13 on
account_summary, because the Daily Balance table’s rows are summed into its
evidence. Under warn that is a spurious warning; under fail it is a lost
document, and the long transaction list the check exists for passes cleanly.
Nine fields of this shape ship in the config library — account_summary,
W2.codes, Payslip.FederalTaxes / StateTaxes / CityTaxes,
Medical-Insurance-Invoice.Charges, PA-Procedure-Log.codes_without_documentation
(a declared subset by definition), PA-Medical-History.chronic_conditions — and
of the narrow arrays in presets with TABLES on, only Transactions genuinely
models table rows. Three further shapes produce the same spurious failure and are
tracked for narrowing: sibling lists whose property counts differ (the
same-width grouping that stops Deposits and Withdrawals accusing each other keys
on equality, so one extra property on one sibling disables it); a nested optional
sub-list, which replaces its parent as the compared target so the parent’s own
completeness supplies the evidence that fails it; and any list nested under a
plain object property, which is never compared at all.
maxItems bounds the evidence, and is the per-field remedy
A declared maxItems on a list field is a ceiling on expected. If the schema
says a list holds at most fifteen rows and extraction returned fifteen, the
extraction is complete by the config author’s own definition, so OCR evidence
above the ceiling is not evidence of a shortfall: a maxItems: 15 list beside a
40-row table is compared against 15 rather than against the table’s 41.
extraction.validation already treats trimming a list to its maxItems as a
correction rather than a loss; this is the same reading.
Three properties of the bound are worth knowing before you rely on it:
A width group is bounded only when every list in it declares a ceiling.
Lists of the same item-property count share the OCR evidence and are judged as
one group, so a single undeclared sibling leaves the group’s legitimate total
unbounded and the evidence is used as-is. When every member declares one, the
group’s ceiling is their sum.
⚠️ A ceiling below 30 rows takes the field out of the check entirely, because
the check only applies where the compared figure is at least 30 rows. Be clear
about what that costs: it is not only that a small list cannot lose 30 rows, it is
that a shortfall relative to the ceiling also goes unreported — 2 rows extracted
against a declared maxItems: 15 is silent, and it was reported before. That is
the price of maxItems being able to take a field out of the check at all, which
is what makes it a usable per-field opt-out for a group-shaped array such as
account_summary, alongside the sibling protection above. Use it deliberately
rather than as a side effect of declaring a small bound.
⚠️ A ceiling lower than the rows a document really holds weakens the check in
proportion, at every size, because the ceiling is the denominator — this is
not confined to the sub-30 case above. On an 800-row statement, 43 extracted rows
are a shortfall against the OCR evidence and are not a shortfall against a
declared maxItems: 80; with maxItems: 100, 50 rows of 500 are silent. So
declare a ceiling your longest expected document can actually reach, and treat
maxItems as a statement about the data rather than as a tuning knob for this
check.
So before setting fail: confirm every array-of-object field in your classes
models table rows rather than an entity group, and that no unrelated table in the
same section shares a width with one of them; where one does, declaring maxItems
on the group-shaped field bounds it. fail is the right setting for a corpus of
long transaction lists, which is the case it was built for. The remaining
narrowing — which is what would let fail be the default — is tracked in
issue #1046.
⚠️ row_shortfall_action: fail is the only thing deciding the outcome here, and
under it a document that lost 95% of its rows fails while one that lost 100%
completes. This is worth knowing before you rely on the setting, because the
uncovered case is the worse one.
A width group in which every list came back empty is not compared against the
OCR evidence at all. It is reported by extraction_incomplete, which is a
warning, and a warning cannot change a document’s status — so a section whose OCR
evidences 800 table rows and whose list came back [] completes, while the same
section returning 43 of those rows fails.
One other setting does reach the empty case, in Advanced mode only: a schema
minItems floor, which the agent’s tool boundary enforces, so [] is rejected
there and a floor the section cannot reach fails it. That is a different mechanism
with a different cost — read
minItems in Simple vs Advanced mode
before reaching for it, because it discards the rows rather than saving them. The
rest of this section describes the behaviour with no such floor in force, which is
every shipped preset.
Both populations are real, and the empty one is larger. Of the 3,631 recorded
COMPLETED benchmark runs that reach this check’s population — the section’s OCR
text evidences at least 30 table rows of the list’s shape — 99 returned zero
rows against 65 with partial loss, so fail catches the smaller of the two.
Why it is left that way. A genuinely empty list is a common, legitimate
outcome — an account with no fees, a statement period with no deposits — and the
check cannot tell it apart from total loss, whereas “43 of 800” has no innocent
reading. Extending the failure to the empty case would therefore fail correct
documents, and the over-attribution described above applies more strongly there,
because an empty list scores 0 against whatever evidence is attributed to it. The
recorded corpus cannot bound the false-failure rate either: it contains no
legitimately-empty-list documents that also carry a same-width table, so the case
needs its own measurement. Both that measurement and the narrowing it depends on
are tracked in
issue #1046.
What to do in the meantime. Treat extraction_incomplete on a class you
expect rows from as equivalent in severity to a row shortfall. It reaches the
document list’s Processing Issues column, the Processing Report tab, and
the tracking table’s sparse HasProcessingIssues attribute, so it is filterable —
which is what has to substitute for document status here.
minItems on a list field — a warning in Simple mode, a hard floor in Advanced
No shipped preset sets minItems; it is something you add.
Transactions:
type: array
# A floor declares that a document under it is a FAILURE, so pick one no
# legitimate document of the class falls below — for a statement corpus whose
# shortest month still carries dozens of rows, a floor well under that count.
# `minItems: 1` is the value to think twice about: it fails every document
# whose list is legitimately empty. Read the Advanced-mode note below before
# setting any floor there.
minItems: 20
items: { … }
minItems is the one completeness signal with no false positives by
construction, because the schema author declared the floor rather than a heuristic
inferring it. What it costs is not the same in the two extraction modes, and the
difference decides how you should pick the number:
Simple (extraction.mode: simple)
Advanced (extraction.mode: advanced)
Where the floor is checked
after extraction, on the returned result
inside the extraction loop, at the agent’s tool boundary — and per shard when the section shards
A list under the floor
raises extraction_list_truncated (warning) and, when extraction.validation.enabled is on (the default since v0.7), a schema violation in metadata.validation / extraction_validation_failed
is rejected; the agent gets a bounded number of correction rounds, and if it still cannot reach the floor the extraction fails
The rows that were extracted
kept, and visible in the section result
discarded — a failed extraction writes no inference_result
Document status
unchanged: COMPLETED
the section fails, so the document fails
In Simple mode minItems buys visibility and nothing else. No processing
issue changes a document’s status by virtue of its severity — including the
error-severity extraction_rows_below_ocr_estimate above, which fails its
section by raising rather than by being an error — and the strongest thing
extraction.validation.fail_action: reject does is record
parsing_succeeded: false, which the Processing Report and the UI’s Processing
Report tab read and the status path does not. So the value is that the loss becomes
something you can filter and alert on, and
extraction.row_shortfall_action: fail above is what decides the outcome.
⚠️ In Advanced mode minItems is a hard floor, and an unreachable one costs the
document. The Pydantic model the agent fills carries the constraint (only scalar
leaves are made nullable for transport, not the list bounds), so a short list is
rejected where it is produced. The correction rounds are whole-section agent turns
and each is billed; when they run out, the section fails with Failed to generate
valid structured output, carrying no rows and no diagnosis of what was short. That
is a worse outcome than row_shortfall_action: fail, which writes the partial
rows, the error-severity issue and the Processing Report first and only then fails
the section.
So, in Advanced mode:
Set a floor you are willing to fail the document on.minItems: 1 on a list
that is sometimes legitimately empty will fail those documents. A floor near the
true row count on a corpus where the agent occasionally falls a row short will
fail those too.
If what you want is a signal rather than a failure, use
extraction.row_shortfall_action — it keeps the data and the explanation.
⚠️ A whole-section floor cannot be satisfied by any shard, and Advanced mode
shards on the shipped defaults. This is the ordinary case rather than an edge
case: max_concurrent_batches ships at 10 — in base-extraction.yaml and in
the Advanced extraction settings the Configuration editor renders — so choosing
Advanced mode is the whole of the opt-in. Sharding then engages whenever the
section exceeds one shard’s budget, which at the shipped max_pages_per_shard: 5
means any section over five pages of ordinary text, and fewer pages when they
are dense enough to fill the shard token budget. Each shard agent’s extraction tool
is built from a model that carries the whole section’s row-count bounds, so the
floor is enforced per shard, not at the merge. A shard sees only its page range, so
a minItems: 100 floor on a 17-page section — four shards at the shipped defaults —
rejects every shard that holds fewer than 100 rows, and a shard over a cover page
holds none at all. The document fails even though it genuinely contains 800 rows.
There is no floor that is both useful and safe here: the only value every shard can
satisfy is no floor. The relaxed per-shard feedback validator, which drops
required and minItems precisely because a shard legitimately holds neither,
governs the agent’s self-correction round rather than the tool boundary that rejects
the call, so it does not rescue this. So if the section shards, do not put
minItems on its lists; use extraction.row_shortfall_action, which is evaluated
once on the merged section and is the only completeness lever here that is
shard-aware.
required behaves differently, and needs no such warning. A shard’s tool does
accept null for a required list or nested object, so a shard covering pages that
contain none of a required table answers null — the answer its instruction asks for
— and is not sent back to correct it. Presence is judged once on the merged section,
against the real class schema, where a null property reads as absent and is reported
by extraction.validation like any other required-property violation. That split is
why a required list is safe on a section that shards and a section-sized minItems
is not: a shard holding none of the rows can satisfy required (with null, or with
[]), and can satisfy no row-count floor at all.
Without minItems, the OCR-row estimate
(extraction_rows_below_ocr_estimate) is what catches a partial Simple-mode list — it
compares the rows extracted with the rows in the section’s OCR tables of the same shape,
so 43 of 800 is reported with no minItems set at all. What neither signal can do is
distinguish a list that returns 10 of 1,200 rows from a document whose OCR shows only
10 table rows from a document that genuinely has 10: with no evidence of the missing
rows, nothing is detectable.
For corpora where large tables are expected — in practice anything beyond ~400 rows
or ~10 pages per document — use Advanced mode, which holds recall 1.000 through
3,200 rows by sharding. Simple mode deliberately does not shard: that is the capability
that distinguishes the two modes. When a Simple-mode section is too large to fit the
model’s input window at all, the run fails (Bedrock’s Input is too long for
requested model); the
failure is raised as ExtractionInputTooLarge with an explanation and the remedy (the
estimated request size, the window, and “use Advanced extraction or split the document”)
in the Step Functions cause and the extraction log, and it is deliberately not retried.
The pre-flight estimate is logged before the call.
Advanced mode: an empty list is retried when the OCR proves there were rows
In Advanced (agentic) mode, one case does not need minItems to be caught.
The OCR pre-flight scan already counts Markdown pipe-table rows in the section, so
when it finds a substantial table (>30 rows) and every declared list field
comes back with no rows, the extraction loop rejects the result and gives the
agent an explicit correction round naming the field and the row count. If some
list is populated, the check stays quiet — the detected tables plausibly belong to
that one, and an empty sibling may be genuinely absent.
This check needs no configuration. It runs on every Advanced-mode section, and
in particular it is not behind extraction.validation.enabled — a guard against
silent data loss that has to be switched on protects nobody who did not already
know to look. (That argument is also why extraction.validation.enabled itself now
defaults to on as of v0.7; this check stays ungated regardless, so explicitly
turning validation off does not also disable a guard whose whole effect is to ask
the model to try again.) Its only effect is one more agent turn — which is billed,
so the worst case is one wasted turn on a document that genuinely has no rows inside
a detected table — and it can never fail a document.
This closes a real failure mode: an agent declined the deterministic table parser
because one column was OCR-corrupted, then returned the whole 100-row list as
null — treating “I cannot map this cleanly” as “therefore no rows”. The
result was schema-valid, scalar accuracy was 1.000, and the section was reported
COMPLETED. The prompt now states the rule explicitly: declining the tool obliges
the agent to extract the table directly, and one unreadable column means that
cell is null, not that the row or the list is dropped.
The Processing Report does not contradict itself over this case. With no
minItems no schema constraint is broken, so the completeness line would
otherwise read ✓ All schema constraints satisfied immediately above the warning
that the list was empty. Instead it reads ⚠ and says which list returned no rows,
how many rows the OCR found, whether the table tool ran, and that setting
minItems would report the same shortfall as a schema violation as well.
Confidence and geometry are appended to extraction results in the
explainability_info format expected by the UI. The format matches the structure
of inference_result, with three attribute shapes:
Simple attributes — a single {confidence, confidence_reason, confidence_threshold, geometry} object.
Group attributes — nested objects, one confidence object per sub-attribute.
List attributes — an array with one confidence object per field per item (assess each item separately, not as an aggregate).
Complete example (all three shapes, with geometry):
Response requirements: match the extraction structure exactly; assess each
list item separately; provide confidence for each sub-attribute of a group; each
assessment includes confidence (0.0–1.0) and optionally confidence_reason;
the system automatically adds confidence_threshold from configuration.
Prefer Advanced (agentic) for production and for complex/nested schemas, strict validation, and large documents/tables.
Write clear attribute descriptions — detail where and how information appears; more specific descriptions yield better extraction (and, in agentic mode, stronger Pydantic validation).
Balance precision vs recall based on whether false positives or false negatives hurt more for your use case.
Optimize few-shot examples — diverse, representative examples covering common variations and edge cases.
Use CachePoint strategically — cache static content, isolate dynamic content, place the tag right before document text.
Optimize image dimensions — original resolution for forms/tables; smaller for simple text at high volume.
Separate classes for very different layouts of the same document type.
Test end-to-end with the full OCR → classification → extraction pipeline.
Choose models by task — Nova Pro for complex few-shot extraction; Claude Haiku for balanced cost; Claude Sonnet for agentic and highest accuracy.
Confidence & geometry
Be specific about high vs low confidence in prompts; include reasoning examples.
Cost management — disable confidence for non-critical classes (enabled: false); start text-only before adding images; monitor token usage.
Model selection — Claude Haiku/Sonnet class models with temperature: 0 for deterministic scoring.
Risk-based thresholds — 0.90+ for critical data, 0.75–0.85 global defaults, per-attribute overrides where needed.
Keep ocr_only geometry (the default) unless you have a specific reason — it is cheaper and more accurate than LLM boxes.
Lower list_batch_size for large lists only if a chunk under-enumerates — it is a ceiling on a derived size, so raising it does not cut the inference count.
Verify extraction.confidence.enabled: true and mode is not off.
Confirm the assessment Lambda deployed successfully.
For agentic + separate/integrated, remember the standalone step is intentionally bypassed (intelligent skip) once extraction writes explainability_info — this is expected, not a failure.
Model produced invalid sequence as part of ToolUse (Advanced extraction)
The configured extraction model emitted a toolUse block Bedrock rejected
mid-stream. The same request on the same model reproduces it, so it is treated as
deterministic rather than transient and is not retried
(#895).
The failure message names the model and lists models measured on this path.
Remedies: reduce extraction.agentic.shard_token_budget /
extraction.agentic.max_pages_per_shard so each call emits less; set
extraction.model (or a class’s x-aws-idp-extraction-model override) to a model
measured on this path; or set extraction.mode: simple, which needs no tool use in
its default configuration. Amazon Nova Lite in particular has not completed the
Advanced path — see
Supported models for agentic extraction.
Inference parameters are not the lever here, despite what AWS’s
Nova tool-use troubleshooting guide
suggests. Greedy decoding — temperature: 0 plus topP: 1 plus topK: 1, the
setting that guide recommends — leaves Nova Lite’s failure rate statistically
unchanged — 0.787 with no topK, 0.812 with topK: 1, 0.838 with the full triple,
across 160 direct calls per arm (Fisher p = 0.68 and p = 1.00; a large improvement
is excluded, an improvement smaller than 18% relative is not). What
determines the outcome is the shape of the request: a schema holding only a list
fails far less often than the same list with sibling scalar properties beside it,
independently of schema size. Measurements, the per-model topK routing table and
the power limits:
Greedy decoding and invalid tool-use sequences.
Template errors
Ensure task_prompt is defined.
Validate placeholder syntax; use exactly one{DOCUMENT_IMAGE} when using images ("found N occurrences, but exactly 1 is required").
Validate OCR quality and the {OCR_TEXT_CONFIDENCE} data.
High costs
Monitor token usage in CloudWatch logs.
Prefer text-only assessment; trim unnecessary prompt context.
Use large-list batching (Simple + separate) rather than one oversized inference.
No bounding boxes generated
In ocr_only/llm_grounded, geometry is advisory — a field with no confident OCR match simply has none. Check that OCR provides geometry (geometryAvailable) and that pageData.json exists for the document.
In llm/llm_grounded, confirm the prompt requests bbox/page data and the model returns valid [x1, y1, x2, y2] in 0–1000 scale with 1-based page numbers.
geometry.mode: off produces no geometry by design.
Invalid coordinates
Ensure LLM boxes are in 0–1000 range; reversed coordinates are auto-corrected, malformed ones dropped.
Confidence threshold / UI issues
Verify confidence_threshold values are between 0.0 and 1.0 and present in explainability_info.
Confirm color coding (green/red/black) and nested-data display.
Key metrics to monitor
InputDocumentsForAssessment, assessment_time_seconds, assessment_parsing_succeeded, and token-consumption logs in CloudWatch.
