Extract Data

ai.extract, Extract structured data from text using AI with a JSON schema Use ai.extract after parsing or collecting text when you need typed JSON back: invoice fields, contract clauses, email metadata, line items, or classifications with supporting fields. Keep the schema as narrow as possible. Enum fields work well for closed choices, and descriptions should explain business meaning, not just repeat the field name. For deterministic calculations after extraction, pass the output into transform.script instead of asking the model to do math. The worker validates the returned JSON against this schema after every provider call. Missing required fields, wrong types, and nested constraint failures fail the step with a schema-validation error; they are never reclassified as ordinary “not found” values. Use provider to select an Eigenpal catalog entry such as anthropic/claude-sonnet-4-6. The optional model field is only a raw provider-specific override (for example a vendor model version) and must not contain an Eigenpal catalog ID. Online validation reports this mix-up before the workflow runs.

Grounding and confidence (on by default)

Grounding is on unless you set grounded: false. Step output stays schema-shaped; provenance lives under a reserved _grounding map keyed by dotted field path (array indexes are decimal segments), so nested values and their grounding line up:

# {{ steps.extract-invoice.output.vendor }}                              -> "Acme Corp"
# {{ steps.extract-invoice.output._grounding.vendor.confidence }}        -> "high"
# {{ steps.extract-invoice.output._grounding.vendor.needsReview }}       -> false
# {{ steps.extract-invoice.output._grounding['line_items.0.amount'].score }} -> 0.92

Each entry carries categorical confidence (high, medium, low), needsReview, optional numeric score, reason, quote, source_span, and — when spatial evidence exists — citations with page/element references. Lineage and the full parsed document graph are never mixed into step JSON; they are stored as optional run sidecars for review and audit surfaces.

Shared grounded extraction (parse → extract)

When input resolves to an object that includes a valid OpenParser ParsedDocument in structured (for example the whole {{ steps.parse.output }} from a layout-aware parse) and your configured provider supports a single strict structured extraction call, Eigenpal runs the same grounded extraction algorithm as OpenParser locally — no call to the hosted OpenParser extract service. On-prem deployments use the same library in the worker. Wire the parse output object, not just .text, to unlock spatial citations and sidecars:

- name: parse
  type: ai.parse
  with:
    input: '{{ input.document }}'
    parseMode: ocr
    ocrModel: openparser-default

- name: extract-invoice
  type: ai.extract
  with:
    input: '{{ steps.parse.output }}'
    schema:
      type: object
      properties:
        vendor:
          type: string
        total:
          type: number
      required: [vendor, total]

Passing only {{ steps.parse.output.text }} still works; extraction and grounding fall back to the text path described below.

Text grounding fallback

When there is no valid ParsedDocument, or the provider cannot run one strict structured call, Eigenpal uses the legacy text grounding pass over the resolved text. That path may set _grounding._degraded: true with a truthful _reason (for example unsupported provider, missing grounding model, or credit quota too low for the second pass). It does not emit lineage sidecars.

high means the value appears verbatim in the source text, medium means it matches approximately or is derived from a highlighted passage the model attested, and low means no supporting source could be located.
Fields at or below the reviewOn threshold carry needsReview: true. Default is low_only; use medium_or_low to also flag approximate matches.
source_span holds character offsets into the resolved text; the run view highlights them in the Grounding panel. Long documents are chunked automatically on the text path.

The text pass uses your workspace default model unless groundingModel picks a different one. Set grounded: true to fail the step when grounding cannot run instead of degrading; grounded: false removes _grounding entirely. Note that grounding confidence measures whether the value is supported in the source the step saw, not whether OCR read the original document correctly. For numeric cross-checks (for example rate times hours equals gross), add a transform.script step.

Current limitations

Spatial citations and lineage sidecars require one canonical ParsedDocument on the extract input. Multiple merged documents are not supported for bbox identity yet.
Legacy text grounding never writes lineage sidecars — only _grounding in step output.
Human review decisions are not yet appended back into lineage.

Configuration

Configuration goes inside the step’s with: block.

string

required

Text content or template expression

object

required

JSON Schema defining the structure to extract

Show schema properties

string

required

record<string, unknown>

required

array<string>

string

Custom prompt template for extraction

string

Provider ID (e.g., “openai-gpt4o”)

string

Model override

Reasoning effort for models that support it. Omit to use the selected model’s default.

integer

Max input tokens. Truncates input text and logs a warning when exceeded. Omit for no limit.

boolean

Grounding is ON by default: each schema field gets a source span + confidence (high=verbatim, medium=fuzzy, low=ungrounded) under a reserved _grounding output key, and fields whose value cannot be located in the source are flagged for human review. Values stay the reliable schema-typed ones. The pass runs through the workspace LLM (any provider) and chunks long documents automatically. Tri-state: unset (default) = on, degrading gracefully to deterministic text alignment (_grounding._degraded: true) if no grounding model is available; true = strict, the step fails when the grounding model cannot be resolved; false = off, no _grounding key at all.

string

Provider/model for the grounding pass. Defaults to the workspace default LLM. Any configured provider works; the pass only fails the step when grounded: true is set explicitly and no model resolves.

Reasoning effort for models that support it. Omit to use the selected model’s default.

array<object>

Optional few-shot examples pinning grounding to verbatim source text per field.

Show groundingExamples properties

string

required

array<object>

required

Show extractions properties

string

required

string

required

"medium_or_low" | "low_only"

Which grounding confidences set needsReview on a field. Default: low_only (only fields whose value could not be located in the source). Use medium_or_low to also flag approximate and derived matches.

Output

Returns record<string, unknown>. Extracted structured data matching the provided schema. Unless grounding is disabled (grounded: false), the output also carries a reserved _grounding map keyed by dotted field path:

_grounding["line_items.0.amount"] = &#123; confidence: high|medium|low, score?, needsReview, reason?, quote?, citations?, source_span }

, plus reserved _degraded: true / _reason markers when grounding could not run the shared algorithm or the EigenPal text pass. Lineage is never included in this JSON.

Get started

Concepts

Workflow steps

Guides & tutorials

Changelog

Grounding and confidence (on by default)

Shared grounded extraction (parse → extract)

Text grounding fallback

Current limitations

Configuration

Output

​Grounding and confidence (on by default)

​Shared grounded extraction (parse → extract)

​Text grounding fallback

​Current limitations

​Configuration

​Output

Grounding and confidence (on by default)

Shared grounded extraction (parse → extract)

Text grounding fallback

Current limitations

Configuration

Output