diff --git a/api.md b/api.md index c076e5b..92b1af2 100644 --- a/api.md +++ b/api.md @@ -64,7 +64,7 @@ Methods: The `client.v2` sub-client targets LandingAI's next-generation ADE gateway, which lives on its own host (`api.ade.[env].landing.ai`) rather than the V1 host (`api.va.[env].landing.ai`). It is **additive**: `client.v2.*` is a separate surface from the top-level `client.*` (V1) methods documented above, and using it does not change any V1 behavior. See the [README](README.md#environments) for environment selection and usage examples. -`client.v2.parse_jobs` and `client.v2.extract_jobs` both return a single, unified `Job` shape, even though the underlying parse/extract job envelopes differ upstream -- `Job.raw` retains the full original envelope as an escape hatch for any field not surfaced on the typed model. +`client.v2.parse_jobs`, `client.v2.extract_jobs`, and `client.v2.ground_jobs` all return a single, unified `Job` shape, even though the underlying parse/extract/ground job envelopes differ upstream -- `Job.raw` retains the full original envelope as an escape hatch for any field not surfaced on the typed model. Types: @@ -77,6 +77,9 @@ from landingai_ade.types.v2 import ( V2ExtractMetadata, V2ExtractResult, V2FileUploadResponse, + V2GroundBilling, + V2GroundMetadata, + V2GroundResult, V2ParseBilling, V2ParseBox, V2ParseElement, @@ -95,8 +98,9 @@ from landingai_ade.types.v2 import ( - Job -- unified job shape: `job_id`, `status` (JobStatus: `pending` / `processing` / `completed` / `failed` / `cancelled`), `created_at`, `completed_at`, `progress`, `result` (a `V2ParseResponse` for parse jobs, a `V2ExtractResult` for extract jobs, or `None` until completion), `error` (JobError), `raw` (the full original envelope as a `dict`), and the `.is_terminal` property. - V2ParseResponse -- `markdown`, `structure`, `grounding`, `metadata` (V2ParseMetadata, which nests V2ParseBilling and carries `output_markdown_chars`, `range_units`, and `openapi_spec`). `structure` is a typed V2ParseStructure tree (`document` → V2ParsePage → V2ParseElement); each node below the root carries its spatial data inline in a V2ParseNodeGrounding (`page`, V2ParseRange, V2ParseBox, normalized page coordinates), and leaf elements additionally carry an `atomic_grounding` list. With `options.inline_markdown`, each node also carries its `markdown` slice. The legacy top-level `grounding` tree (V2ParseGrounding → `V2ParseGroundingPage` → `V2ParseGroundingElement` → `V2ParseGroundingEntry`) is retained for older gateway responses. Element `type`/page `status` are permissive strings and unknown keys are retained. -- V2ExtractResult -- `extraction`, `extraction_metadata`, `markdown`, `output_ref`, `metadata` (V2ExtractMetadata, which carries `model_version`, `range_units`, `openapi_spec`, and nests V2ExtractBilling with `input_markdown_chars` / `output_extraction_chars`). +- V2ExtractResult -- `extraction`, `extraction_metadata`, `markdown`, `output_ref`, `schema_violation_error` (set when `strict=False` and the schema had unextractable fields), `warnings`, and `metadata` (V2ExtractMetadata, which carries `model_version`, `input_markdown_chars`, `output_extraction_chars`, `range_units`, `openapi_spec`, and nests V2ExtractBilling). - V2FileUploadResponse -- `file_ref`. +- V2GroundResult -- `grounding` (a tree mirroring the input `extraction_metadata`, each `{value, ranges}` leaf replaced by the list of `structure` blocks its ranges overlap) and `metadata` (V2GroundMetadata: `job_id`, `duration_ms`, `openapi_spec`, and nested V2GroundBilling). Methods: @@ -115,18 +119,29 @@ Methods: Synchronous extract. `schema` accepts a pydantic `BaseModel` subclass, a `dict`, or a JSON-encoded string -- all are coerced to a JSON Schema object. Provide exactly one of `markdown` or `markdown_url`. `strict=True` rejects schemas with unsupported fields (HTTP 422) instead of silently pruning them. Raises `V2SyncTimeoutError` on a 504; use `extract_jobs` for long-running documents. -- client.v2.extract_jobs.create(\*, schema, markdown=..., markdown_url=..., model=..., strict=..., service_tier=...) -> Job +- client.v2.extract_jobs.create(\*, schema, markdown=..., markdown_url=..., model=..., strict=..., output_save_url=..., service_tier=...) -> Job - client.v2.extract_jobs.get(job_id) -> Job - client.v2.extract_jobs.list(\*, page=..., page_size=..., status=...) -> JobList[Job] - client.v2.extract_jobs.wait(job_id, \*, timeout=600, poll_interval=None, raise_on_failure=False) -> Job Same polling/timeout semantics as `parse_jobs.wait`. Extract jobs have no `cancelled` status, so `raise_on_failure` only ever triggers on `failed`. +- client.v2.ground(\*, extraction_metadata, structure) -> V2GroundResult + + Synchronous ground. Maps each extracted field back to the `structure` blocks it was quoted from by overlapping `extraction_metadata` ranges against every block's inline `grounding.range`. Both `extraction_metadata` (e.g. `client.v2.extract(...).extraction_metadata`) and `structure` (e.g. `client.v2.parse(...).structure`) accept a `dict` or a pydantic model. Block ids resolve only against the `structure` supplied here, so pass the parse result the extraction actually came from. Raises `V2SyncTimeoutError` on a 504; use `ground_jobs` for long-running inputs. + +- client.v2.ground_jobs.create(\*, extraction_metadata, structure, output_save_url=...) -> Job +- client.v2.ground_jobs.get(job_id) -> Job +- client.v2.ground_jobs.list(\*, page=..., page_size=..., status=...) -> JobList[Job] +- client.v2.ground_jobs.wait(job_id, \*, timeout=600, poll_interval=None, raise_on_failure=False) -> Job + + Same polling/timeout semantics as `parse_jobs.wait`. Ground jobs have no `cancelled` status, so `raise_on_failure` only ever triggers on `failed`. `output_save_url` (create only) delivers the result out-of-band and reports `output_url` on the completed job instead of an inline `result`. + - client.v2.files.upload(\*, file) -> str Stages a file's bytes on the ADE data plane and returns a `file_ref` string for use in job inputs. Served on the ADE host under `/v1/files` (not `/v2/...`). Raises `LandingAiadeError` if the response has no `file_ref`. Notes: -- `parse_jobs.list` / `extract_jobs.list` both return a `JobList` (a `list[Job]` subclass) carrying pagination metadata: `.has_more`, `.org_id`, `.page`, `.page_size`. +- `parse_jobs.list` / `extract_jobs.list` / `ground_jobs.list` all return a `JobList` (a `list[Job]` subclass) carrying pagination metadata: `.has_more`, `.org_id`, `.page`, `.page_size`. - All `client.v2.*` methods accept the usual `extra_headers`, `extra_query`, `extra_body`, and `timeout` overrides; sync methods additionally accept `save_to` (parse/extract only, not the job-creation methods) to write the response to disk, mirroring V1's `save_to`. diff --git a/docs/v2-testing.md b/docs/v2-testing.md index c93f224..4b6f04b 100644 --- a/docs/v2-testing.md +++ b/docs/v2-testing.md @@ -8,8 +8,8 @@ what to check when the upstream spec (`specs/v2-aide.json`) changes. | Layer | Location | What it covers | | --- | --- | --- | -| Response models | `tests/test_v2_types.py` | Deserialization of `V2ParseResponse` / `V2ExtractResult` and their nested models from plain dicts, including unknown-key tolerance. | -| Job normalization | `tests/test_v2_normalize.py` | `normalize_parse_job` / `normalize_extract_job`: envelope → unified `Job` (status, timestamps, `result`, `error`). | +| Response models | `tests/test_v2_types.py` | Deserialization of `V2ParseResponse` / `V2ExtractResult` / `V2GroundResult` and their nested models from plain dicts, including unknown-key tolerance. | +| Job normalization | `tests/test_v2_normalize.py` | `normalize_parse_job` / `normalize_extract_job` / `normalize_ground_job`: envelope → unified `Job` (status, timestamps, `result`, `error`). | | Resource wiring | `tests/api_resources/v2/` | `respx`-mocked HTTP: host routing, multipart/JSON bodies, options serialization, job polling. No network. | | Live smoke | `tests/contract/test_v2_smoke.py` | End-to-end calls against staging (marked `contract`; skipped unless `LANDINGAI_ADE_STAGING_APIKEY` is set). | @@ -54,15 +54,42 @@ responses omit it in favor of the inline `grounding` above. `POST /v2/extract` (and the completed `extract_jobs` result) returns a `V2ExtractResult` with `extraction`, `extraction_metadata`, `markdown`, -`output_ref` (set when the output was delivered out-of-band), and `metadata` -(`V2ExtractMetadata`): `job_id`, `model_version`, `duration_ms`, `doc_id`, -`credit_usage`, `range_units`, `openapi_spec`, and `billing` (`V2ExtractBilling` -with `input_markdown_chars` / `output_extraction_chars`). +`output_ref` (deprecated; renamed to `schema_violation_error` upstream), +`schema_violation_error` (set when `options.strict` is false and the schema had +fields the model could not extract — the extraction is partial), `warnings` +(non-fatal warnings), and `metadata` (`V2ExtractMetadata`): `job_id`, +`model_version`, `duration_ms`, `doc_id`, `input_markdown_chars`, +`output_extraction_chars`, `credit_usage` (deprecated), `range_units`, +`openapi_spec`, and `billing` (`V2ExtractBilling`). The `input_markdown_chars` / +`output_extraction_chars` char counts moved from `billing` onto `metadata` +upstream; both are retained on `V2ExtractBilling` for backward compatibility. + +The async `extract_jobs.create` also accepts `output_save_url` (async jobs only): +when set, the finished result is delivered to that URL and the completed job +reports `output_url` (on `Job.raw`) instead of an inline `result`. + +## Current ground-response shape + +`POST /v2/ground` (and the completed `ground_jobs` result) returns a +`V2GroundResult` — a pure, stateless join that maps each extracted field back to +the `structure` blocks it was quoted from: + +- `grounding` — a tree mirroring the input `extraction_metadata`: nested objects + and arrays keep their shape, and each `{value, ranges}` leaf is replaced by the + list of `structure` blocks its ranges overlap (block ids resolve only against + the `structure` supplied in the request). +- `metadata` (`V2GroundMetadata`) — `job_id`, `duration_ms`, `openapi_spec`, and + `billing` (`V2GroundBilling`). + +`client.v2.ground(...)` takes `extraction_metadata` and `structure`, each of which +accepts a plain `dict` or a pydantic model (so a parse response's `.structure` can +be passed directly). `ground_jobs.create` additionally accepts `output_save_url`. ## Async job envelopes -`normalize_parse_job` and `normalize_extract_job` fold the upstream job envelopes -into the unified `Job`. Both are tolerant of field-name drift: +`normalize_parse_job`, `normalize_extract_job`, and `normalize_ground_job` fold +the upstream job envelopes into the unified `Job`. All are tolerant of field-name +drift: - The parse response lives under `result` (older envelopes used `data`). - Failures arrive as a structured `error` object (`{code, message}`); older parse diff --git a/specs/_generated/v2_models.py b/specs/_generated/v2_models.py index 834a21b..486f1d5 100644 --- a/specs/_generated/v2_models.py +++ b/specs/_generated/v2_models.py @@ -6,10 +6,13 @@ from enum import Enum from typing import Any, Literal, Optional, Union -from pydantic import BaseModel, Field, RootModel +from pydantic import BaseModel, ConfigDict, Field, RootModel class BaseElementOptions(BaseModel): + model_config = ConfigDict( + extra='forbid', + ) markdown: Optional[bool] = Field(True, title='Markdown') @@ -66,7 +69,18 @@ class Type(Enum): scan_code = 'scan_code' +class ErrorResponse(BaseModel): + code: str = Field( + ..., + description='Stable snake_case error code (e.g. ``validation_error``, ``unknown_model_version``, ``invalid_url``, ``invalid_api_key``, ``rate_limit_exceeded``).', + ) + message: str = Field(..., description='Human-readable detail.') + + class FigureOptions(BaseModel): + model_config = ConfigDict( + extra='forbid', + ) markdown: Optional[bool] = Field(True, title='Markdown') @@ -176,6 +190,9 @@ class Format(Enum): class TableOptions(BaseModel): + model_config = ConfigDict( + extra='forbid', + ) format: Optional[Format] = Field('html', title='Format') markdown: Optional[bool] = Field(True, title='Markdown') @@ -186,16 +203,6 @@ class V2Billing(BaseModel): charged. """ - input_markdown_chars: Optional[int] = Field( - None, - description='Characters (Unicode code points) in the input markdown as submitted — the input basis of the credit charge. Extract responses only.', - title='Input Markdown Chars', - ) - output_extraction_chars: Optional[int] = Field( - None, - description='Characters in the serialized extraction output — the output basis of the credit charge. Extract responses only.', - title='Output Extraction Chars', - ) service_tier: Optional[ServiceTier] = Field( None, description='The service tier the request ran in: `standard` or `priority`. A sync request reports `priority` (same lane, same price).', @@ -215,9 +222,6 @@ class V2ExtractMetadata(BaseModel): None, description='Billing summary: the service tier the request ran in and the credits charged.', ) - credit_usage: Optional[float] = Field( - 0.0, description='Credits billed for this request.', title='Credit Usage' - ) doc_id: Optional[str] = Field( None, description='Present when the input markdown contained a ```` comment (embedded by ``POST /v2/parse``). Links this extract call to the originating parse job.', @@ -228,6 +232,11 @@ class V2ExtractMetadata(BaseModel): description='End-to-end request duration in milliseconds.', title='Duration Ms', ) + input_markdown_chars: Optional[int] = Field( + None, + description='Characters (Unicode code points) in the input markdown as submitted — the input basis of the credit charge.', + title='Input Markdown Chars', + ) job_id: str = Field( ..., description='Gateway job id (workflow id). Matches the ``x-request-id`` the gateway minted for this request and the billing row id in vision-agent.', @@ -240,6 +249,11 @@ class V2ExtractMetadata(BaseModel): ..., description='URL of the OpenAPI spec covering this API, for inspection and client generation.', ) + output_extraction_chars: Optional[int] = Field( + None, + description='Characters in the serialized extraction output — the output basis of the credit charge.', + title='Output Extraction Chars', + ) range_units: Literal['unicode_codepoints'] = Field( ..., description='Units of every `range` offset in the response. Always `"unicode_codepoints"` (Unicode code points into `markdown`). Declared explicitly so consumers know how to slice the string — e.g. JavaScript strings are UTF-16, so a naive `.slice()` drifts when the markdown contains astral characters.', @@ -252,13 +266,41 @@ class V2ExtractOptions(BaseModel): Extraction options (``docs/extract-v2-proposal.md`` → Options). """ + model_config = ConfigDict( + extra='forbid', + ) strict: Optional[bool] = Field( False, - description='When ``true``, returns HTTP 422 if the schema contains fields the model cannot extract. When ``false`` (default), unsupported fields are skipped and extraction continues.', + description='When ``true``, a schema containing fields the model cannot extract fails with a validation error — HTTP 422 on the sync route, or a failed job (``status: "failed"``) on the async ``/jobs`` route. When ``false`` (default), unsupported fields are skipped and extraction continues.', title='Strict', ) +class V2GroundMetadata(BaseModel): + """ + Response metadata for a v2 ground call. + """ + + billing: Optional[V2Billing] = Field( + None, + description='Billing summary: the service tier the request ran in and the credits charged.', + ) + duration_ms: int = Field( + ..., + description='End-to-end request duration in milliseconds.', + title='Duration Ms', + ) + job_id: str = Field( + ..., + description='Gateway job id (workflow id). Matches the ``x-request-id`` the gateway minted for this request.', + title='Job Id', + ) + openapi_spec: str = Field( + ..., + description='URL of the OpenAPI spec covering this API, for inspection and client generation.', + ) + + class V2WorkflowMetadata(BaseModel): """ Response metadata for a v2 workflow call @@ -269,11 +311,6 @@ class V2WorkflowMetadata(BaseModel): None, description='Billing summary: the service tier the request ran in and the credits charged.', ) - credit_usage: Optional[float] = Field( - 0.0, - description='Combined credits billed for this request (parse + extract).', - title='Credit Usage', - ) duration_ms: int = Field( ..., description='Total end-to-end duration across all stages in milliseconds.', @@ -340,12 +377,15 @@ class WorkflowStepOptions(BaseModel): ) strict: Optional[bool] = Field( False, - description='When ``true``, returns 422 if the schema contains fields the model cannot extract. When ``false``, unsupported fields are skipped.', + description='When ``true``, a schema containing fields the model cannot extract fails with a validation error — HTTP 422 on the sync route, or a failed job on the async ``/jobs`` route. When ``false``, unsupported fields are skipped.', title='Strict', ) class BlocksOptions(BaseModel): + model_config = ConfigDict( + extra='forbid', + ) attestation: Optional[BaseElementOptions] = None card: Optional[BaseElementOptions] = None figure: Optional[FigureOptions] = None diff --git a/specs/v2-aide.json b/specs/v2-aide.json index 8a324d5..d28e2a5 100644 --- a/specs/v2-aide.json +++ b/specs/v2-aide.json @@ -2,6 +2,7 @@ "components": { "schemas": { "BaseElementOptions": { + "additionalProperties": false, "properties": { "markdown": { "default": true, @@ -13,6 +14,7 @@ "type": "object" }, "BlocksOptions": { + "additionalProperties": false, "properties": { "attestation": { "$ref": "#/components/schemas/BaseElementOptions" @@ -257,7 +259,26 @@ "title": "Element", "type": "object" }, + "ErrorResponse": { + "properties": { + "code": { + "description": "Stable snake_case error code (e.g. ``validation_error``, ``unknown_model_version``, ``invalid_url``, ``invalid_api_key``, ``rate_limit_exceeded``).", + "type": "string" + }, + "message": { + "description": "Human-readable detail.", + "type": "string" + } + }, + "required": [ + "code", + "message" + ], + "title": "ErrorResponse", + "type": "object" + }, "FigureOptions": { + "additionalProperties": false, "properties": { "markdown": { "default": true, @@ -606,6 +627,7 @@ "type": "object" }, "TableOptions": { + "additionalProperties": false, "properties": { "format": { "default": "html", @@ -628,32 +650,6 @@ "V2Billing": { "description": "Billing summary: the service tier the request ran in and the credits\ncharged.", "properties": { - "input_markdown_chars": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "default": null, - "description": "Characters (Unicode code points) in the input markdown as submitted — the input basis of the credit charge. Extract responses only.", - "title": "Input Markdown Chars" - }, - "output_extraction_chars": { - "anyOf": [ - { - "type": "integer" - }, - { - "type": "null" - } - ], - "default": null, - "description": "Characters in the serialized extraction output — the output basis of the credit charge. Extract responses only.", - "title": "Output Extraction Chars" - }, "service_tier": { "anyOf": [ { @@ -702,12 +698,6 @@ ], "description": "Billing summary: the service tier the request ran in and the credits charged." }, - "credit_usage": { - "default": 0.0, - "description": "Credits billed for this request.", - "title": "Credit Usage", - "type": "number" - }, "doc_id": { "anyOf": [ { @@ -726,6 +716,19 @@ "title": "Duration Ms", "type": "integer" }, + "input_markdown_chars": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Characters (Unicode code points) in the input markdown as submitted — the input basis of the credit charge.", + "title": "Input Markdown Chars" + }, "job_id": { "description": "Gateway job id (workflow id). Matches the ``x-request-id`` the gateway minted for this request and the billing row id in vision-agent.", "title": "Job Id", @@ -740,6 +743,19 @@ "description": "URL of the OpenAPI spec covering this API, for inspection and client generation.", "type": "string" }, + "output_extraction_chars": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Characters in the serialized extraction output — the output basis of the credit charge.", + "title": "Output Extraction Chars" + }, "range_units": { "const": "unicode_codepoints", "default": "unicode_codepoints", @@ -759,11 +775,12 @@ "type": "object" }, "V2ExtractOptions": { + "additionalProperties": false, "description": "Extraction options (``docs/extract-v2-proposal.md`` → Options).", "properties": { "strict": { "default": false, - "description": "When ``true``, returns HTTP 422 if the schema contains fields the model cannot extract. When ``false`` (default), unsupported fields are skipped and extraction continues.", + "description": "When ``true``, a schema containing fields the model cannot extract fails with a validation error — HTTP 422 on the sync route, or a failed job (``status: \"failed\"``) on the async ``/jobs`` route. When ``false`` (default), unsupported fields are skipped and extraction continues.", "title": "Strict", "type": "boolean" } @@ -771,6 +788,43 @@ "title": "V2ExtractOptions", "type": "object" }, + "V2GroundMetadata": { + "description": "Response metadata for a v2 ground call.", + "properties": { + "billing": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2Billing" + }, + { + "type": "null" + } + ], + "description": "Billing summary: the service tier the request ran in and the credits charged." + }, + "duration_ms": { + "description": "End-to-end request duration in milliseconds.", + "title": "Duration Ms", + "type": "integer" + }, + "job_id": { + "description": "Gateway job id (workflow id). Matches the ``x-request-id`` the gateway minted for this request.", + "title": "Job Id", + "type": "string" + }, + "openapi_spec": { + "description": "URL of the OpenAPI spec covering this API, for inspection and client generation.", + "type": "string" + } + }, + "required": [ + "job_id", + "duration_ms", + "openapi_spec" + ], + "title": "V2GroundMetadata", + "type": "object" + }, "V2WorkflowMetadata": { "description": "Response metadata for a v2 workflow call\n(``docs/pipeline-v2-proposal.md`` → Top-level metadata).", "properties": { @@ -785,12 +839,6 @@ ], "description": "Billing summary: the service tier the request ran in and the credits charged." }, - "credit_usage": { - "default": 0.0, - "description": "Combined credits billed for this request (parse + extract).", - "title": "Credit Usage", - "type": "number" - }, "duration_ms": { "description": "Total end-to-end duration across all stages in milliseconds.", "title": "Duration Ms", @@ -910,7 +958,7 @@ }, "strict": { "default": false, - "description": "When ``true``, returns 422 if the schema contains fields the model cannot extract. When ``false``, unsupported fields are skipped.", + "description": "When ``true``, a schema containing fields the model cannot extract fails with a validation error — HTTP 422 on the sync route, or a failed job on the async ``/jobs`` route. When ``false``, unsupported fields are skipped.", "title": "Strict", "type": "boolean" } @@ -1170,9 +1218,9 @@ }, "metadata": { "$ref": "#/components/schemas/V2ExtractMetadata", - "description": "Request metadata (job_id, model_version, duration_ms, doc_id, credit_usage)." + "description": "Request metadata (job_id, model_version, duration_ms, doc_id, billing)." }, - "output_ref": { + "schema_violation_error": { "anyOf": [ { "type": "string" @@ -1182,7 +1230,17 @@ } ], "default": null, - "title": "Output Ref" + "description": "Set when ``options.strict`` is false and the schema contained fields the model could not extract — the extraction is partial.", + "title": "Schema Violation Error" + }, + "warnings": { + "description": "Non-fatal warnings emitted during extraction.", + "items": { + "additionalProperties": true, + "type": "object" + }, + "title": "Warnings", + "type": "array" } }, "required": [ @@ -1202,11 +1260,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE Extract", @@ -1337,11 +1395,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE List Extract Jobs", @@ -1409,6 +1467,19 @@ "default": null, "description": "Extraction options (``strict``). Omit for defaults." }, + "output_save_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time.", + "title": "Output Save Url" + }, "schema": { "additionalProperties": true, "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions.", @@ -1505,6 +1576,19 @@ "default": null, "description": "Extraction options (``strict``). Omit for defaults. JSON-serialized string in form data." }, + "output_save_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time. JSON-serialized string in form data.", + "title": "Output Save Url" + }, "schema": { "additionalProperties": true, "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions. JSON-serialized string in form data.", @@ -1580,6 +1664,16 @@ } }, "description": "Job created" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." } }, "summary": "ADE Extract Jobs", @@ -1638,8 +1732,18 @@ "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, + "output_url": { + "description": "The URL the result was delivered to. Present once the job has ``completed`` and ``output_save_url`` was set, instead of inline ``result``.", + "type": [ + "string", + "null" + ] + }, "progress": { - "description": "Best-effort progress, present while processing only for jobs that report it." + "description": "Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.", + "maximum": 1, + "minimum": 0, + "type": "number" }, "result": { "anyOf": [ @@ -1665,9 +1769,9 @@ }, "metadata": { "$ref": "#/components/schemas/V2ExtractMetadata", - "description": "Request metadata (job_id, model_version, duration_ms, doc_id, credit_usage)." + "description": "Request metadata (job_id, model_version, duration_ms, doc_id, billing)." }, - "output_ref": { + "schema_violation_error": { "anyOf": [ { "type": "string" @@ -1677,7 +1781,17 @@ } ], "default": null, - "title": "Output Ref" + "description": "Set when ``options.strict`` is false and the schema contained fields the model could not extract — the extraction is partial.", + "title": "Schema Violation Error" + }, + "warnings": { + "description": "Non-fatal warnings emitted during extraction.", + "items": { + "additionalProperties": true, + "type": "object" + }, + "title": "Warnings", + "type": "array" } }, "required": [ @@ -1693,7 +1807,7 @@ "type": "null" } ], - "description": "Present once status is ``completed``." + "description": "Present once status is ``completed`` and ``output_save_url`` was not set. When ``output_save_url`` was set, the result is delivered there and ``output_url`` is returned instead." }, "status": { "enum": [ @@ -1711,15 +1825,25 @@ }, "description": "Job status / result" }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Not found (e.g. no such job)." + }, "422": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE Get Extract Jobs", @@ -1728,79 +1852,79 @@ ] } }, - "/v2/parse": { + "/v2/ground": { "post": { - "description": "Parse a document and return the parse response inline.", - "operationId": "parse_run_sync", + "description": "Map extracted fields to the document blocks they were quoted from. Takes the extraction_metadata from an extract call and the structure tree from the parse call the markdown came from, and returns, for every extracted field, the overlapping blocks with their ids, page numbers, and bounding boxes. Runs synchronously and returns the result inline.", + "operationId": "v2-ground_run_sync", "requestBody": { "content": { - "multipart/form-data": { + "application/json": { "schema": { + "description": "Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body.\n\nA pure, stateless join: each ``extraction_metadata`` leaf's ``ranges``\n(char offsets into the markdown both artifacts were produced from) is\noverlapped against the ``grounding.range`` carried on every ``structure``\nblock, and the matching blocks are returned. Nothing is stored server-side,\nso block ids in the response resolve only against the ``structure`` tree\nsupplied here — pairing an extraction with the parse result it actually\ncame from is the caller's responsibility.", "properties": { - "document": { - "description": "The file to parse. The file must be a PDF or image; see the list of [supported file types](https://docs.landing.ai/dpt3/file-types). Provide either `document` or `document_url`, not both.", - "format": "binary", - "type": "string" - }, - "document_url": { - "description": "A publicly accessible URL to the file to parse. The file must be a PDF or image; see the list of [supported file types](https://docs.landing.ai/dpt3/file-types). Provide either `document` or `document_url`, not both.", - "type": "string" - }, - "model": { - "description": "The DPT-3 model snapshot to use for this request. Accepts a dated snapshot (for example, `dpt-3-pro-20260710`), the `dpt-3-pro-latest` alias, or the bare `dpt-3-pro` family name (equivalent to `dpt-3-pro-latest`). Defaults to the latest DPT-3 Pro snapshot.", - "type": "string" - }, - "options": { - "description": "Optional object that customizes the parse. Use it to select which pages to process, adjust how content appears in the Markdown, or control how much detail the response includes. Sent as a JSON-serialized string in form data.", - "properties": { - "atomic_grounding": { - "default": true, - "description": "Include the fine-grained `atomic_grounding` array on leaf elements. Set `false` to omit the field entirely from every node.", - "title": "Atomic Grounding", - "type": "boolean" - }, - "blocks": { - "$ref": "#/components/schemas/BlocksOptions" - }, - "inline_markdown": { - "default": false, - "description": "Include each node's slice of the document `markdown` inline as a `markdown` field on every structure node: the document root, each page, and each element (including table cells). `atomic_grounding` entries do not carry it.", - "title": "Inline Markdown", - "type": "boolean" - }, - "pages": { - "anyOf": [ - { - "items": { - "type": "integer" - }, - "type": "array" - }, + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", + "example": { + "invoice_number": { + "ranges": [ { - "type": "null" + "end": 31, + "start": 13 } ], - "default": null, - "title": "Pages" - }, - "password": { - "anyOf": [ - { - "type": "string" - }, + "value": "INV-042" + } + }, + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.", + "title": "Structure", + "type": "object" + } + }, + "required": [ + "extraction_metadata", + "structure" + ], + "title": "V2GroundRequest", + "type": "object" + } + }, + "multipart/form-data": { + "schema": { + "properties": { + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", + "example": { + "invoice_number": { + "ranges": [ { - "type": "null" + "end": 31, + "start": 13 } ], - "default": null, - "description": "Password for encrypted PDFs. Not currently supported — providing a value returns a 422 error; decrypt the file before uploading.", - "title": "Password" + "value": "INV-042" } }, - "title": "ParseOptions", + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.", + "title": "Structure", "type": "object" } }, + "required": [ + "extraction_metadata", + "structure" + ], "type": "object" } } @@ -1812,11 +1936,570 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/ParseResponse" + "description": "Result returned by V2GroundOperationWorkflow — the ``/v2/ground``\nresponse body.\n\n``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and\narrays keep their shape, and each ``{value, ranges}`` leaf is replaced by\nthe list of structure blocks its ranges overlap (the block-hit shape\ndocumented on the field). It is NOT a flat map — a nested schema field like\n``issuer.name`` resolves to ``grounding[\"issuer\"][\"name\"]``.", + "properties": { + "grounding": { + "additionalProperties": true, + "description": "A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", + "example": { + "invoice_number": [ + { + "atomic_grounding": [], + "block_id": "text-1", + "grounding": { + "box": { + "xmax": 0.42, + "xmin": 0.1, + "ymax": 0.15, + "ymin": 0.12 + }, + "page": 1, + "range": { + "end": 31, + "start": 13 + } + }, + "type": "text" + } + ] + }, + "title": "Grounding", + "type": "object" + }, + "metadata": { + "$ref": "#/components/schemas/V2GroundMetadata", + "description": "Request metadata (job_id, duration_ms, credit_usage)." + } + }, + "required": [ + "grounding", + "metadata" + ], + "title": "V2GroundResult", + "type": "object" } } }, - "description": "The parse response" + "description": "v2-ground result" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE Ground", + "tags": [ + "Ground" + ] + } + }, + "/v2/ground/jobs": { + "get": { + "description": "List your Ground jobs, newest first.", + "operationId": "v2-ground_list_jobs", + "parameters": [ + { + "description": "Page number (0-indexed).", + "in": "query", + "name": "page", + "required": false, + "schema": { + "default": 0, + "description": "Page number (0-indexed).", + "minimum": 0, + "title": "Page", + "type": "integer" + } + }, + { + "description": "Number of items per page.", + "in": "query", + "name": "page_size", + "required": false, + "schema": { + "default": 10, + "description": "Number of items per page.", + "maximum": 100, + "minimum": 1, + "title": "Page Size", + "type": "integer" + } + }, + { + "description": "Filter by job status.", + "in": "query", + "name": "status", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Filter by job status.", + "title": "Status" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "properties": { + "has_more": { + "type": "boolean" + }, + "jobs": { + "items": { + "properties": { + "completed_at": { + "type": [ + "string", + "null" + ] + }, + "created_at": { + "type": [ + "string", + "null" + ] + }, + "failure_reason": { + "type": [ + "string", + "null" + ] + }, + "job_id": { + "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "type": "string" + }, + "model_version": { + "type": [ + "string", + "null" + ] + }, + "status": { + "enum": [ + "pending", + "processing", + "completed", + "failed" + ], + "type": "string" + } + }, + "type": "object" + }, + "type": "array" + }, + "page": { + "type": "integer" + }, + "page_size": { + "type": "integer" + } + }, + "type": "object" + } + } + }, + "description": "The caller's jobs, newest first" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE List Ground Jobs", + "tags": [ + "Ground" + ] + }, + "post": { + "description": "Map extracted fields to the document blocks they were quoted from. Takes the extraction_metadata from an extract call and the structure tree from the parse call the markdown came from, and returns, for every extracted field, the overlapping blocks with their ids, page numbers, and bounding boxes. Runs asynchronously and returns a job ID; use it to poll for status and retrieve the result once processing completes.", + "operationId": "v2-ground_create_job", + "requestBody": { + "content": { + "application/json": { + "schema": { + "description": "Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body.\n\nA pure, stateless join: each ``extraction_metadata`` leaf's ``ranges``\n(char offsets into the markdown both artifacts were produced from) is\noverlapped against the ``grounding.range`` carried on every ``structure``\nblock, and the matching blocks are returned. Nothing is stored server-side,\nso block ids in the response resolve only against the ``structure`` tree\nsupplied here — pairing an extraction with the parse result it actually\ncame from is the caller's responsibility.", + "properties": { + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", + "example": { + "invoice_number": { + "ranges": [ + { + "end": 31, + "start": 13 + } + ], + "value": "INV-042" + } + }, + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.", + "title": "Structure", + "type": "object" + } + }, + "required": [ + "extraction_metadata", + "structure" + ], + "title": "V2GroundRequest", + "type": "object" + } + }, + "multipart/form-data": { + "schema": { + "properties": { + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", + "example": { + "invoice_number": { + "ranges": [ + { + "end": 31, + "start": 13 + } + ], + "value": "INV-042" + } + }, + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.", + "title": "Structure", + "type": "object" + } + }, + "required": [ + "extraction_metadata", + "structure" + ], + "type": "object" + } + } + }, + "required": true + }, + "responses": { + "202": { + "content": { + "application/json": { + "schema": { + "properties": { + "created_at": { + "type": [ + "string", + "null" + ] + }, + "job_id": { + "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "type": "string" + }, + "status": { + "enum": [ + "pending", + "processing", + "completed", + "failed" + ], + "type": "string" + } + }, + "type": "object" + } + } + }, + "description": "Job created" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE Ground Jobs", + "tags": [ + "Ground" + ] + } + }, + "/v2/ground/jobs/{job_id}": { + "get": { + "description": "Get the status of an async Ground job, including its result once the job has completed.", + "operationId": "v2-ground_get_job", + "parameters": [ + { + "description": "The identifier of the job to retrieve, as returned by the create-job request.", + "in": "path", + "name": "job_id", + "required": true, + "schema": { + "description": "The identifier of the job to retrieve, as returned by the create-job request.", + "title": "Job Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "properties": { + "completed_at": { + "description": "Present once the job is terminal.", + "type": "string" + }, + "created_at": { + "type": [ + "string", + "null" + ] + }, + "error": { + "description": "Present once status is ``failed``.", + "properties": { + "code": { + "description": "Stable error code (``internal_error`` when unmapped).", + "type": "string" + }, + "message": { + "type": "string" + } + }, + "type": "object" + }, + "job_id": { + "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "type": "string" + }, + "progress": { + "description": "Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.", + "maximum": 1, + "minimum": 0, + "type": "number" + }, + "result": { + "anyOf": [ + { + "description": "Result returned by V2GroundOperationWorkflow — the ``/v2/ground``\nresponse body.\n\n``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and\narrays keep their shape, and each ``{value, ranges}`` leaf is replaced by\nthe list of structure blocks its ranges overlap (the block-hit shape\ndocumented on the field). It is NOT a flat map — a nested schema field like\n``issuer.name`` resolves to ``grounding[\"issuer\"][\"name\"]``.", + "properties": { + "grounding": { + "additionalProperties": true, + "description": "A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", + "example": { + "invoice_number": [ + { + "atomic_grounding": [], + "block_id": "text-1", + "grounding": { + "box": { + "xmax": 0.42, + "xmin": 0.1, + "ymax": 0.15, + "ymin": 0.12 + }, + "page": 1, + "range": { + "end": 31, + "start": 13 + } + }, + "type": "text" + } + ] + }, + "title": "Grounding", + "type": "object" + }, + "metadata": { + "$ref": "#/components/schemas/V2GroundMetadata", + "description": "Request metadata (job_id, duration_ms, credit_usage)." + } + }, + "required": [ + "grounding", + "metadata" + ], + "title": "V2GroundResult", + "type": "object" + }, + { + "type": "null" + } + ], + "description": "Present once status is ``completed``." + }, + "status": { + "enum": [ + "pending", + "processing", + "completed", + "failed" + ], + "type": "string" + } + }, + "type": "object" + } + } + }, + "description": "Job status / result" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Not found (e.g. no such job)." + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE Get Ground Jobs", + "tags": [ + "Ground" + ] + } + }, + "/v2/parse": { + "post": { + "description": "Parse a document and return the parse response inline.", + "operationId": "parse_run_sync", + "requestBody": { + "content": { + "multipart/form-data": { + "schema": { + "properties": { + "document": { + "description": "The file to parse. The file must be a PDF or image; see the list of [supported file types](https://docs.landing.ai/dpt3/file-types). Provide either `document` or `document_url`, not both.", + "format": "binary", + "type": "string" + }, + "document_url": { + "description": "A publicly accessible URL to the file to parse. The file must be a PDF or image; see the list of [supported file types](https://docs.landing.ai/dpt3/file-types). Provide either `document` or `document_url`, not both.", + "type": "string" + }, + "model": { + "description": "The DPT-3 model snapshot to use for this request. Accepts a dated snapshot (for example, `dpt-3-pro-20260710`), the `dpt-3-pro-latest` alias, or the bare `dpt-3-pro` family name (equivalent to `dpt-3-pro-latest`). Defaults to the latest DPT-3 Pro snapshot.", + "type": "string" + }, + "options": { + "additionalProperties": false, + "description": "Optional object that customizes the parse. Use it to select which pages to process, adjust how content appears in the Markdown, or control how much detail the response includes. Sent as a JSON-serialized string in form data.", + "properties": { + "atomic_grounding": { + "default": true, + "description": "Include the fine-grained `atomic_grounding` array on leaf elements. Set `false` to omit the field entirely from every node.", + "title": "Atomic Grounding", + "type": "boolean" + }, + "blocks": { + "$ref": "#/components/schemas/BlocksOptions" + }, + "inline_markdown": { + "default": false, + "description": "Include each node's slice of the document `markdown` inline as a `markdown` field on every structure node: the document root, each page, and each element (including table cells). `atomic_grounding` entries do not carry it.", + "title": "Inline Markdown", + "type": "boolean" + }, + "pages": { + "anyOf": [ + { + "items": { + "type": "integer" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Pages" + }, + "password": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Password for encrypted PDFs. Not currently supported — providing a value returns a 422 error; decrypt the file before uploading.", + "title": "Password" + } + }, + "title": "ParseOptions", + "type": "object" + } + }, + "type": "object" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ParseResponse" + } + } + }, + "description": "The parse response" }, "206": { "content": { @@ -1832,11 +2515,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE Parse", @@ -1934,6 +2617,13 @@ "description": "The unique identifier for the parse job. Format: ``-<26-character Crockford base32 ULID>`` matching ``^(parse|extract)-[0-9a-hjkmnp-tv-z]{26}$``. Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, + "model_version": { + "description": "The model snapshot used to parse the document.", + "type": [ + "string", + "null" + ] + }, "status": { "description": "The job's current status: ``pending``, ``processing``, ``completed``, or ``failed``.", "enum": [ @@ -1943,13 +2633,6 @@ "failed" ], "type": "string" - }, - "version": { - "description": "The model snapshot used to parse the document.", - "type": [ - "string", - "null" - ] } }, "type": "object" @@ -1975,11 +2658,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE List Parse Jobs", @@ -2009,6 +2692,7 @@ "type": "string" }, "options": { + "additionalProperties": false, "description": "Optional object that customizes the parse. Use it to select which pages to process, adjust how content appears in the Markdown, or control how much detail the response includes. Sent as a JSON-serialized string in form data.", "properties": { "atomic_grounding": { @@ -2115,6 +2799,16 @@ } }, "description": "Job created" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." } }, "summary": "ADE Parse Jobs", @@ -2214,17 +2908,24 @@ "description": "Job status / result" }, "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, "description": "Job not found" }, "422": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "ADE Get Parse Jobs", @@ -2272,7 +2973,7 @@ "description": "Optional projection map. Keys are caller-chosen response field names; values are ``$output.....`` references. Omitted → full results tree returned under ``output``.", "example": { "revenue": "$output.parse-extract.extract.extraction.revenue", - "revenue_box": "$output.parse-extract.ground.revenue" + "revenue_blocks": "$output.parse-extract.ground.revenue" }, "title": "Output" }, @@ -2339,7 +3040,7 @@ "description": "Optional projection map. Keys are caller-chosen response field names; values are ``$output.....`` references. Omitted → full results tree returned under ``output``. JSON-serialized string in form data.", "example": { "revenue": "$output.parse-extract.extract.extraction.revenue", - "revenue_box": "$output.parse-extract.ground.revenue" + "revenue_blocks": "$output.parse-extract.ground.revenue" }, "title": "Output" }, @@ -2381,29 +3082,17 @@ "content": { "application/json": { "schema": { - "description": "Result returned by V2WorkflowOperationWorkflow.\n\n``output`` is keyed by step name (or by caller-chosen projection keys when\n``output`` was provided in the request). For the ``\"parse-extract\"`` pipeline\nwith no projection, the keys are ``\"parse-extract\"`` → ``{parse, extract, ground}``:\n\n* ``output[\"parse-extract\"][\"parse\"]`` — full parse response (markdown,\n structure with per-node grounding inline, metadata).\n* ``output[\"parse-extract\"][\"extract\"]`` — extraction + ``extraction_metadata``\n with ``{value, ranges}`` leaves.\n* ``output[\"parse-extract\"][\"ground\"]`` — a tree MIRRORING\n ``extraction_metadata``: each ``{value, ranges}`` leaf is replaced by its\n resolved ``[{page, box}]`` list (or ``null`` for a synthesised value). Nested\n objects and arrays keep their shape, so ``ground.issuer.name`` parallels\n ``extraction_metadata.issuer.name``. Resolved server-side — no client-side\n span lookup needed.\n\nReferences into the results use the ``$output.....`` grammar\n(see ``output`` field of the request).", + "description": "Result returned by V2WorkflowOperationWorkflow.\n\n``output`` is keyed by step name (or by caller-chosen projection keys when\n``output`` was provided in the request). For the ``\"parse-extract\"`` pipeline\nwith no projection, the keys are ``\"parse-extract\"`` → ``{parse, extract, ground}``:\n\n* ``output[\"parse-extract\"][\"parse\"]`` — full parse response (markdown,\n structure with per-node grounding inline, metadata).\n* ``output[\"parse-extract\"][\"extract\"]`` — extraction + ``extraction_metadata``\n with ``{value, ranges}`` leaves.\n* ``output[\"parse-extract\"][\"ground\"]`` — a tree MIRRORING\n ``extraction_metadata``: each ``{value, ranges}`` leaf is replaced by its\n resolved block-hit list (``[{block_id, type, parent_id?, grounding,\n atomic_grounding?}]`` — the same shape ``/v2/ground`` returns; see\n ``V2GroundResult.grounding`` for the full field-level contract), or\n ``null`` for a synthesised value. Nested objects and arrays keep their\n shape, so ``ground.issuer.name`` parallels\n ``extraction_metadata.issuer.name``. Resolved server-side — no\n client-side span lookup needed.\n\nReferences into the results use the ``$output.....`` grammar\n(see ``output`` field of the request).", "properties": { "metadata": { "$ref": "#/components/schemas/V2WorkflowMetadata", - "description": "Request metadata (job_id, duration_ms, credit_usage)." + "description": "Request metadata (job_id, duration_ms, billing)." }, "output": { "additionalProperties": true, "description": "Step results keyed by step name, or a caller-defined projection. For ``parse-extract`` with no projection: ``{\"parse-extract\": {parse, extract, ground}}``.", "title": "Output", "type": "object" - }, - "output_ref": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "default": null, - "title": "Output Ref" } }, "required": [ @@ -2421,11 +3110,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "Run v2-workflow (sync)", @@ -2555,11 +3244,11 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "List v2-workflow jobs", @@ -2604,7 +3293,7 @@ "description": "Optional projection map. Keys are caller-chosen response field names; values are ``$output.....`` references. Omitted → full results tree returned under ``output``.", "example": { "revenue": "$output.parse-extract.extract.extraction.revenue", - "revenue_box": "$output.parse-extract.ground.revenue" + "revenue_blocks": "$output.parse-extract.ground.revenue" }, "title": "Output" }, @@ -2686,7 +3375,7 @@ "description": "Optional projection map. Keys are caller-chosen response field names; values are ``$output.....`` references. Omitted → full results tree returned under ``output``. JSON-serialized string in form data.", "example": { "revenue": "$output.parse-extract.extract.extraction.revenue", - "revenue_box": "$output.parse-extract.ground.revenue" + "revenue_blocks": "$output.parse-extract.ground.revenue" }, "title": "Output" }, @@ -2769,6 +3458,16 @@ } }, "description": "Job created" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." } }, "summary": "Create v2-workflow job", @@ -2828,34 +3527,25 @@ "type": "string" }, "progress": { - "description": "Best-effort progress, present while processing only for jobs that report it." + "description": "Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.", + "maximum": 1, + "minimum": 0, + "type": "number" }, "result": { "anyOf": [ { - "description": "Result returned by V2WorkflowOperationWorkflow.\n\n``output`` is keyed by step name (or by caller-chosen projection keys when\n``output`` was provided in the request). For the ``\"parse-extract\"`` pipeline\nwith no projection, the keys are ``\"parse-extract\"`` → ``{parse, extract, ground}``:\n\n* ``output[\"parse-extract\"][\"parse\"]`` — full parse response (markdown,\n structure with per-node grounding inline, metadata).\n* ``output[\"parse-extract\"][\"extract\"]`` — extraction + ``extraction_metadata``\n with ``{value, ranges}`` leaves.\n* ``output[\"parse-extract\"][\"ground\"]`` — a tree MIRRORING\n ``extraction_metadata``: each ``{value, ranges}`` leaf is replaced by its\n resolved ``[{page, box}]`` list (or ``null`` for a synthesised value). Nested\n objects and arrays keep their shape, so ``ground.issuer.name`` parallels\n ``extraction_metadata.issuer.name``. Resolved server-side — no client-side\n span lookup needed.\n\nReferences into the results use the ``$output.....`` grammar\n(see ``output`` field of the request).", + "description": "Result returned by V2WorkflowOperationWorkflow.\n\n``output`` is keyed by step name (or by caller-chosen projection keys when\n``output`` was provided in the request). For the ``\"parse-extract\"`` pipeline\nwith no projection, the keys are ``\"parse-extract\"`` → ``{parse, extract, ground}``:\n\n* ``output[\"parse-extract\"][\"parse\"]`` — full parse response (markdown,\n structure with per-node grounding inline, metadata).\n* ``output[\"parse-extract\"][\"extract\"]`` — extraction + ``extraction_metadata``\n with ``{value, ranges}`` leaves.\n* ``output[\"parse-extract\"][\"ground\"]`` — a tree MIRRORING\n ``extraction_metadata``: each ``{value, ranges}`` leaf is replaced by its\n resolved block-hit list (``[{block_id, type, parent_id?, grounding,\n atomic_grounding?}]`` — the same shape ``/v2/ground`` returns; see\n ``V2GroundResult.grounding`` for the full field-level contract), or\n ``null`` for a synthesised value. Nested objects and arrays keep their\n shape, so ``ground.issuer.name`` parallels\n ``extraction_metadata.issuer.name``. Resolved server-side — no\n client-side span lookup needed.\n\nReferences into the results use the ``$output.....`` grammar\n(see ``output`` field of the request).", "properties": { "metadata": { "$ref": "#/components/schemas/V2WorkflowMetadata", - "description": "Request metadata (job_id, duration_ms, credit_usage)." + "description": "Request metadata (job_id, duration_ms, billing)." }, "output": { "additionalProperties": true, "description": "Step results keyed by step name, or a caller-defined projection. For ``parse-extract`` with no projection: ``{\"parse-extract\": {parse, extract, ground}}``.", "title": "Output", "type": "object" - }, - "output_ref": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "default": null, - "title": "Output Ref" } }, "required": [ @@ -2887,15 +3577,25 @@ }, "description": "Job status / result" }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Not found (e.g. no such job)." + }, "422": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/HTTPValidationError" + "$ref": "#/components/schemas/ErrorResponse" } } }, - "description": "Validation Error" + "description": "Request validation failed." } }, "summary": "Get v2-workflow job", diff --git a/src/landingai_ade/resources/v2/_normalize.py b/src/landingai_ade/resources/v2/_normalize.py index 619f84c..70d2431 100644 --- a/src/landingai_ade/resources/v2/_normalize.py +++ b/src/landingai_ade/resources/v2/_normalize.py @@ -6,9 +6,9 @@ from ..._types import StrBytesIntFloat from ..._utils import parse_datetime -from ...types.v2 import Job, JobError, JobStatus, V2ExtractResult, V2ParseResponse +from ...types.v2 import Job, JobError, JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse -__all__ = ["normalize_parse_job", "normalize_extract_job"] +__all__ = ["normalize_parse_job", "normalize_extract_job", "normalize_ground_job"] def _ts(value: Optional[Union[datetime, StrBytesIntFloat]]) -> Optional[datetime]: @@ -97,3 +97,30 @@ def normalize_extract_job(raw: Mapping[str, Any]) -> Job: error=error, raw=dict(raw), ) + + +def normalize_ground_job(raw: Mapping[str, Any]) -> Job: + status = _status(raw) + payload = raw.get("result") + # Build leniently (like the sync-response path) so unexpected upstream drift + # doesn't fail construction. + result = V2GroundResult.construct(**cast(Dict[str, Any], payload)) if isinstance(payload, Mapping) else None + + error = None + err = raw.get("error") + if isinstance(err, Mapping): + err = cast(Dict[str, Any], err) + error = JobError(code=err.get("code"), message=err.get("message")) + elif raw.get("failure_reason"): # ground *list* uses failure_reason + error = JobError(message=str(raw["failure_reason"])) + + return Job( + job_id=str(raw["job_id"]), + status=status, + created_at=_ts(raw.get("created_at")), + completed_at=_ts(raw.get("completed_at")), + progress=_progress(raw.get("progress")), + result=result, + error=error, + raw=dict(raw), + ) diff --git a/src/landingai_ade/resources/v2/extract.py b/src/landingai_ade/resources/v2/extract.py index dd53b0a..baf0a6c 100644 --- a/src/landingai_ade/resources/v2/extract.py +++ b/src/landingai_ade/resources/v2/extract.py @@ -178,6 +178,7 @@ def create( markdown_url: Optional[str] | Omit = omit, model: Optional[str] | Omit = omit, strict: Optional[bool] | Omit = omit, + output_save_url: Optional[str] | Omit = omit, service_tier: Optional[Literal["standard", "priority"]] | Omit = omit, # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. # The extra values given here take precedence over values defined on the client or passed to this method. @@ -207,6 +208,10 @@ def create( False, prune unsupported fields and continue. Sent as `options.strict`. + output_save_url: URL the result should be saved to (e.g. a presigned S3 PUT + URL) instead of being returned inline. Async jobs only. When set, the + completed job reports `output_url` instead of an inline `result`. + service_tier: Service tier for the job: ``standard`` or ``priority``. extra_headers: Send extra headers @@ -218,6 +223,8 @@ def create( timeout: Override the client-level default timeout for this request, in seconds """ body = _build_extract_body(schema, markdown, markdown_url, model, strict, service_tier) + if is_given(output_save_url) and output_save_url is not None: + body["output_save_url"] = output_save_url raw = self._post( self._v2_url("/v2/extract/jobs"), body=body, @@ -319,6 +326,7 @@ async def create( markdown_url: Optional[str] | Omit = omit, model: Optional[str] | Omit = omit, strict: Optional[bool] | Omit = omit, + output_save_url: Optional[str] | Omit = omit, service_tier: Optional[Literal["standard", "priority"]] | Omit = omit, # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. # The extra values given here take precedence over values defined on the client or passed to this method. @@ -329,6 +337,8 @@ async def create( ) -> Job: """Async mirror of `ExtractJobsResource.create`. See there for full documentation.""" body = _build_extract_body(schema, markdown, markdown_url, model, strict, service_tier) + if is_given(output_save_url) and output_save_url is not None: + body["output_save_url"] = output_save_url raw = await self._post( self._v2_url("/v2/extract/jobs"), body=body, diff --git a/src/landingai_ade/resources/v2/ground.py b/src/landingai_ade/resources/v2/ground.py new file mode 100644 index 0000000..bcc68c3 --- /dev/null +++ b/src/landingai_ade/resources/v2/ground.py @@ -0,0 +1,359 @@ +from __future__ import annotations + +import time +from typing import Any, Dict, Union, Mapping, Callable, Optional, cast + +import httpx +from pydantic import BaseModel + +from ._base import DEFAULT_WAIT_TIMEOUT, JobList, V2ResourceMixin, poll_until_terminal, apoll_until_terminal +from ..._types import Body, Omit, Query, Headers, NotGiven, omit, not_given +from ..._utils import is_given +from ..._compat import model_dump +from ...types.v2 import Job, V2GroundResult +from ._normalize import normalize_ground_job +from ..._resource import SyncAPIResource, AsyncAPIResource +from ..._exceptions import APIStatusError +from ..._base_client import make_request_options +from ...lib.v2_errors import raise_if_sync_timeout + +__all__ = ["GroundResource", "AsyncGroundResource", "GroundJobsResource", "AsyncGroundJobsResource"] + + +def _as_object(value: Union[Mapping[str, object], BaseModel]) -> Dict[str, Any]: + """Coerce a mapping or a pydantic model (e.g. a `V2ParseResponse.structure`) to a plain dict.""" + if isinstance(value, BaseModel): + return model_dump(value) + return dict(value) + + +def _build_ground_body( + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + output_save_url: object = omit, +) -> Dict[str, Any]: + body: Dict[str, Any] = { + "extraction_metadata": _as_object(extraction_metadata), + "structure": _as_object(structure), + } + if is_given(output_save_url) and output_save_url is not None: + body["output_save_url"] = output_save_url + return body + + +class GroundResource(V2ResourceMixin, SyncAPIResource): + def run( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2GroundResult: + """Map extracted fields to the document blocks they were quoted from against + the V2 (ADE) `/v2/ground` endpoint (JSON body). + + A pure, stateless join: each `extraction_metadata` leaf's `ranges` are + overlapped against the `grounding.range` carried on every `structure` + block, and the matching blocks are returned. Block ids in the response + resolve only against the `structure` tree supplied here, so pairing an + extraction with the parse result it actually came from is the caller's + responsibility. + + Raises `V2SyncTimeoutError` when the server times out the synchronous + request (HTTP 504); use the async jobs route for long-running inputs in + that case. + + Args: + extraction_metadata: The `extraction_metadata` tree returned by + `POST /v2/extract` (or `client.v2.extract(...).extraction_metadata`). + Accepts a dict or a pydantic model. + + structure: The `structure` tree from the parse response the extraction was + produced from (e.g. `client.v2.parse(...).structure`). Accepts a dict or + a pydantic model. + + extra_headers: Send extra headers + + extra_query: Add additional query parameters to the request + + extra_body: Add additional JSON properties to the request + + timeout: Override the client-level default timeout for this request, in seconds + """ + body = _build_ground_body(extraction_metadata, structure) + try: + return self._post( + self._v2_url("/v2/ground"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2GroundResult, + ) + except APIStatusError as exc: + raise_if_sync_timeout(exc) + raise + + +class AsyncGroundResource(V2ResourceMixin, AsyncAPIResource): + async def run( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2GroundResult: + """Async mirror of `GroundResource.run`. See there for full documentation.""" + body = _build_ground_body(extraction_metadata, structure) + try: + return await self._post( + self._v2_url("/v2/ground"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2GroundResult, + ) + except APIStatusError as exc: + raise_if_sync_timeout(exc) + raise + + +class GroundJobsResource(V2ResourceMixin, SyncAPIResource): + def create( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + output_save_url: Optional[str] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Create an asynchronous ground job against `/v2/ground/jobs`. + + Returns a normalized `Job` immediately (typically `pending`). Poll for + completion via `.get(job_id)`, or block until the job is terminal with + `.wait(job_id)`. + + Args: + extraction_metadata: The `extraction_metadata` tree returned by + `POST /v2/extract`. Accepts a dict or a pydantic model. + + structure: The `structure` tree from the parse response the extraction was + produced from. Accepts a dict or a pydantic model. + + output_save_url: URL the result should be saved to (e.g. a presigned S3 PUT + URL) instead of being returned inline. When set, the completed job reports + `output_url` instead of an inline `result`. + + extra_headers: Send extra headers + + extra_query: Add additional query parameters to the request + + extra_body: Add additional JSON properties to the request + + timeout: Override the client-level default timeout for this request, in seconds + """ + body = _build_ground_body(extraction_metadata, structure, output_save_url) + raw = self._post( + self._v2_url("/v2/ground/jobs"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_ground_job(cast(Mapping[str, Any], raw)) + + def get( + self, + job_id: str, + *, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Get the current status of an async ground job by `job_id`.""" + if not job_id: + raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") + raw = self._get( + self._v2_url(f"/v2/ground/jobs/{job_id}"), + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_ground_job(cast(Mapping[str, Any], raw)) + + def list( + self, + *, + page: int | Omit = omit, + page_size: int | Omit = omit, + status: Optional[str] | Omit = omit, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> JobList: + """List async ground jobs associated with your API key, newest first.""" + query = { + key: value + for key, value in {"page": page, "page_size": page_size, "status": status}.items() + if is_given(value) and value is not None + } + raw = self._get( + self._v2_url("/v2/ground/jobs"), + options=make_request_options( + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + query=query, + ), + cast_to=cast("type[Any]", object), + ) + env = cast(Mapping[str, Any], raw) + jobs = [normalize_ground_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] + return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) + + def wait( + self, + job_id: str, + *, + timeout: float = DEFAULT_WAIT_TIMEOUT, + poll_interval: Optional[float] = None, + raise_on_failure: bool = False, + _monotonic: Optional[Callable[[], float]] = None, + ) -> Job: + """Block, polling `.get(job_id)` with backoff, until the job is terminal. + + Raises `JobWaitTimeoutError` if `timeout` seconds elapse before the job + reaches a terminal state, and `JobFailedError` if `raise_on_failure` is + set and the job ends failed with an error attached. Ground jobs have no + `cancelled` status. + + `_monotonic` is a test seam for injecting a fake clock; production + callers should leave it unset (defaults to `time.monotonic`). + """ + return poll_until_terminal( + lambda: self.get(job_id), + monotonic=_monotonic or time.monotonic, + sleep=self._sleep, + timeout=timeout, + poll_interval=poll_interval, + raise_on_failure=raise_on_failure, + ) + + +class AsyncGroundJobsResource(V2ResourceMixin, AsyncAPIResource): + async def create( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + output_save_url: Optional[str] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Async mirror of `GroundJobsResource.create`. See there for full documentation.""" + body = _build_ground_body(extraction_metadata, structure, output_save_url) + raw = await self._post( + self._v2_url("/v2/ground/jobs"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_ground_job(cast(Mapping[str, Any], raw)) + + async def get( + self, + job_id: str, + *, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Async mirror of `GroundJobsResource.get`. See there for full documentation.""" + if not job_id: + raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") + raw = await self._get( + self._v2_url(f"/v2/ground/jobs/{job_id}"), + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_ground_job(cast(Mapping[str, Any], raw)) + + async def list( + self, + *, + page: int | Omit = omit, + page_size: int | Omit = omit, + status: Optional[str] | Omit = omit, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> JobList: + """Async mirror of `GroundJobsResource.list`. See there for full documentation.""" + query = { + key: value + for key, value in {"page": page, "page_size": page_size, "status": status}.items() + if is_given(value) and value is not None + } + raw = await self._get( + self._v2_url("/v2/ground/jobs"), + options=make_request_options( + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + query=query, + ), + cast_to=cast("type[Any]", object), + ) + env = cast(Mapping[str, Any], raw) + jobs = [normalize_ground_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] + return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) + + async def wait( + self, + job_id: str, + *, + timeout: float = DEFAULT_WAIT_TIMEOUT, + poll_interval: Optional[float] = None, + raise_on_failure: bool = False, + _monotonic: Optional[Callable[[], float]] = None, + ) -> Job: + """Async mirror of `GroundJobsResource.wait`; sleeps via `anyio.sleep` instead of blocking.""" + return await apoll_until_terminal( + lambda: self.get(job_id), + monotonic=_monotonic or time.monotonic, + timeout=timeout, + poll_interval=poll_interval, + raise_on_failure=raise_on_failure, + ) diff --git a/src/landingai_ade/resources/v2/v2.py b/src/landingai_ade/resources/v2/v2.py index befa049..17b67b5 100644 --- a/src/landingai_ade/resources/v2/v2.py +++ b/src/landingai_ade/resources/v2/v2.py @@ -10,12 +10,13 @@ from ._base import V2ResourceMixin from ..._types import Body, Omit, Query, Headers, NotGiven, FileTypes, omit, not_given from ..._compat import cached_property -from ...types.v2 import V2ExtractResult, V2ParseResponse +from ...types.v2 import V2GroundResult, V2ExtractResult, V2ParseResponse from ..._resource import SyncAPIResource, AsyncAPIResource if TYPE_CHECKING: from .files import FilesResource, AsyncFilesResource from .parse import ParseResource, ParseJobsResource, AsyncParseResource, AsyncParseJobsResource + from .ground import GroundResource, GroundJobsResource, AsyncGroundResource, AsyncGroundJobsResource from .extract import ExtractResource, ExtractJobsResource, AsyncExtractResource, AsyncExtractJobsResource __all__ = ["V2Resource", "AsyncV2Resource"] @@ -24,7 +25,7 @@ class V2Resource(SyncAPIResource, V2ResourceMixin): """Container for the V2 (ADE) surface: ``client.v2.``. - ``files``, ``parse``, and ``extract`` are wired up; each sub-resource does its + ``files``, ``parse``, ``extract``, and ``ground`` are wired up; each sub-resource does its own lazy import inside its cached property body -- mirroring ``LandingAIADE.parse_jobs`` -- so that this module keeps importing standalone regardless of which sub-resources exist yet. Remaining job-polling resources @@ -121,6 +122,40 @@ def extract( timeout=timeout, ) + @cached_property + def _ground(self) -> GroundResource: + from .ground import GroundResource + + return GroundResource(self._client) + + @cached_property + def ground_jobs(self) -> GroundJobsResource: + from .ground import GroundJobsResource + + return GroundJobsResource(self._client) + + def ground( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2GroundResult: + """Ground extracted fields to document blocks synchronously. See ``GroundResource.run`` for full documentation.""" + return self._ground.run( + extraction_metadata=extraction_metadata, + structure=structure, + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + ) + class AsyncV2Resource(AsyncAPIResource, V2ResourceMixin): """Async mirror of :class:`V2Resource`.""" @@ -214,3 +249,37 @@ async def extract( extra_body=extra_body, timeout=timeout, ) + + @cached_property + def _ground(self) -> AsyncGroundResource: + from .ground import AsyncGroundResource + + return AsyncGroundResource(self._client) + + @cached_property + def ground_jobs(self) -> AsyncGroundJobsResource: + from .ground import AsyncGroundJobsResource + + return AsyncGroundJobsResource(self._client) + + async def ground( + self, + *, + extraction_metadata: Union[Mapping[str, object], BaseModel], + structure: Union[Mapping[str, object], BaseModel], + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2GroundResult: + """Async mirror of :meth:`V2Resource.ground`.""" + return await self._ground.run( + extraction_metadata=extraction_metadata, + structure=structure, + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + ) diff --git a/src/landingai_ade/types/v2/__init__.py b/src/landingai_ade/types/v2/__init__.py index 281aa23..54fd8e6 100644 --- a/src/landingai_ade/types/v2/__init__.py +++ b/src/landingai_ade/types/v2/__init__.py @@ -16,6 +16,11 @@ V2ParseGroundingEntry as V2ParseGroundingEntry, V2ParseGroundingElement as V2ParseGroundingElement, ) +from .ground_response import ( + V2GroundResult as V2GroundResult, + V2GroundBilling as V2GroundBilling, + V2GroundMetadata as V2GroundMetadata, +) from .extract_response import ( V2ExtractResult as V2ExtractResult, V2ExtractBilling as V2ExtractBilling, diff --git a/src/landingai_ade/types/v2/extract_response.py b/src/landingai_ade/types/v2/extract_response.py index 82cd25d..3743917 100644 --- a/src/landingai_ade/types/v2/extract_response.py +++ b/src/landingai_ade/types/v2/extract_response.py @@ -1,6 +1,6 @@ from __future__ import annotations -from typing import Dict, Optional +from typing import Dict, List, Optional from ..._models import BaseModel @@ -29,8 +29,16 @@ class V2ExtractMetadata(BaseModel): model_version: Optional[str] = None duration_ms: int doc_id: Optional[str] = None + # Deprecated: superseded by `billing`; retained for backward compatibility + # and populated only by older gateway responses. credit_usage: float = 0.0 billing: Optional[V2ExtractBilling] = None + # Characters (code points) in the input markdown as submitted -- the input + # basis of the credit charge (moved here from `billing` upstream). + input_markdown_chars: Optional[int] = None + # Characters in the serialized extraction output -- the output basis of the + # credit charge (moved here from `billing` upstream). + output_extraction_chars: Optional[int] = None # Units of every `range` offset in the response (always "unicode_codepoints"). range_units: Optional[str] = None # URL of the OpenAPI spec covering this API. @@ -42,6 +50,11 @@ class V2ExtractResult(BaseModel): extraction_metadata: Dict[str, object] markdown: str metadata: V2ExtractMetadata - # Present when the extraction output was delivered out-of-band (e.g. to a - # ZDR save URL) instead of inline. + # Deprecated: renamed to `schema_violation_error` upstream; retained for + # backward compatibility and populated only by older gateway responses. output_ref: Optional[str] = None + # Set when `options.strict` is false and the schema contained fields the + # model could not extract -- the extraction is partial. + schema_violation_error: Optional[str] = None + # Non-fatal warnings emitted during extraction. + warnings: Optional[List[Dict[str, object]]] = None diff --git a/src/landingai_ade/types/v2/ground_response.py b/src/landingai_ade/types/v2/ground_response.py new file mode 100644 index 0000000..2029d7a --- /dev/null +++ b/src/landingai_ade/types/v2/ground_response.py @@ -0,0 +1,33 @@ +from __future__ import annotations + +from typing import Dict, Optional + +from ..._models import BaseModel + +__all__ = ["V2GroundBilling", "V2GroundMetadata", "V2GroundResult"] + + +class V2GroundBilling(BaseModel): + """Billing summary: the service tier the request ran in and the credits charged.""" + + service_tier: Optional[str] = None + total_credits: Optional[float] = None + + +class V2GroundMetadata(BaseModel): + """Response metadata for a v2 ground call.""" + + job_id: str + duration_ms: int + # URL of the OpenAPI spec covering this API. + openapi_spec: Optional[str] = None + billing: Optional[V2GroundBilling] = None + + +class V2GroundResult(BaseModel): + # A tree mirroring the input `extraction_metadata`: each `{value, ranges}` + # leaf is replaced by the list of `structure` blocks its ranges overlap. Not + # a flat map -- a nested field like `issuer.name` resolves to + # `grounding["issuer"]["name"]`. + grounding: Dict[str, object] + metadata: V2GroundMetadata diff --git a/tests/api_resources/v2/test_extract.py b/tests/api_resources/v2/test_extract.py index 67ec1fc..c476cab 100644 --- a/tests/api_resources/v2/test_extract.py +++ b/tests/api_resources/v2/test_extract.py @@ -145,6 +145,41 @@ def test_extract_sync_parses_billing_metadata() -> None: assert result.metadata.billing.total_credits == 12.5 +@respx.mock +def test_extract_sync_parses_char_counts_and_warnings() -> None: + # `input_markdown_chars`/`output_extraction_chars` moved onto `metadata` + # (from `billing`) upstream, and `schema_violation_error`/`warnings` were + # added to the result. + client = LandingAIADE(apikey=APIKEY) + body: Dict[str, Any] = dict(EXTRACT_BODY) + metadata: Dict[str, Any] = dict(EXTRACT_BODY["metadata"]) + metadata["input_markdown_chars"] = 42 + metadata["output_extraction_chars"] = 7 + body["metadata"] = metadata + body["schema_violation_error"] = "unsupported field skipped" + body["warnings"] = [{"code": "partial", "message": "heads up"}] + respx.post("https://api.ade.landing.ai/v2/extract").mock(return_value=httpx.Response(200, json=body)) + result = client.v2.extract(schema={"type": "object"}, markdown="x") + assert result.metadata.input_markdown_chars == 42 + assert result.metadata.output_extraction_chars == 7 + assert result.schema_violation_error == "unsupported field skipped" + assert result.warnings is not None and result.warnings[0]["code"] == "partial" + + +def test_extract_job_create_sends_output_save_url(monkeypatch: pytest.MonkeyPatch) -> None: + # The async job create body carries `output_save_url` (async jobs only). + client = LandingAIADE(apikey=APIKEY) + captured: Dict[str, Any] = {} + + def fake_post(path: str, *, cast_to: Any, body: Any = None, options: Any = None, **kwargs: Any) -> Any: # noqa: ARG001 + captured["body"] = body + return {"job_id": "e1", "status": "pending"} + + monkeypatch.setattr(client.v2.extract_jobs, "_post", fake_post) + client.v2.extract_jobs.create(schema={"type": "object"}, markdown="x", output_save_url="https://example.com/put") + assert captured["body"]["output_save_url"] == "https://example.com/put" + + @respx.mock def test_extract_job_get_failed_maps_error_object() -> None: client = LandingAIADE(apikey=APIKEY) diff --git a/tests/api_resources/v2/test_ground.py b/tests/api_resources/v2/test_ground.py new file mode 100644 index 0000000..8828076 --- /dev/null +++ b/tests/api_resources/v2/test_ground.py @@ -0,0 +1,164 @@ +from __future__ import annotations + +import json +from typing import Any, Dict + +import httpx +import respx +import pytest + +from landingai_ade import LandingAIADE +from landingai_ade.types.v2 import JobStatus, V2GroundResult +from landingai_ade.lib.v2_errors import V2SyncTimeoutError + +APIKEY = "My Apikey" + +EXTRACTION_METADATA: Dict[str, Any] = { + "invoice_number": {"value": "INV-042", "ranges": [{"start": 13, "end": 31}]}, +} +STRUCTURE: Dict[str, Any] = {"type": "document", "children": []} +GROUND_BODY: Dict[str, Any] = { + "grounding": { + "invoice_number": [ + { + "block_id": "text-1", + "type": "text", + "grounding": {"page": 1, "range": {"start": 13, "end": 31}}, + } + ] + }, + "metadata": {"job_id": "ground-1", "duration_ms": 5}, +} + + +@respx.mock +def test_ground_sync_json_body() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/ground").mock(return_value=httpx.Response(200, json=GROUND_BODY)) + result = client.v2.ground(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) + assert isinstance(result, V2GroundResult) + assert result.metadata.job_id == "ground-1" + assert "invoice_number" in result.grounding + req = json.loads(route.calls.last.request.content) + assert req["extraction_metadata"]["invoice_number"]["value"] == "INV-042" + assert req["structure"]["type"] == "document" + assert route.calls.last.request.headers["content-type"].startswith("application/json") + + +@respx.mock +def test_ground_sync_parses_billing_metadata() -> None: + client = LandingAIADE(apikey=APIKEY) + body: Dict[str, Any] = dict(GROUND_BODY) + metadata: Dict[str, Any] = dict(GROUND_BODY["metadata"]) + metadata["billing"] = {"service_tier": "priority", "total_credits": 2.5} + body["metadata"] = metadata + respx.post("https://api.ade.landing.ai/v2/ground").mock(return_value=httpx.Response(200, json=body)) + result = client.v2.ground(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) + assert result.metadata.billing is not None + assert result.metadata.billing.service_tier == "priority" + assert result.metadata.billing.total_credits == 2.5 + + +@respx.mock +def test_ground_sync_504() -> None: + client = LandingAIADE(apikey=APIKEY, max_retries=0) + respx.post("https://api.ade.landing.ai/v2/ground").mock(return_value=httpx.Response(504, json={"detail": "x"})) + with pytest.raises(V2SyncTimeoutError): + client.v2.ground(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) + + +@respx.mock +def test_ground_job_create_and_get() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.post("https://api.ade.landing.ai/v2/ground/jobs").mock( + return_value=httpx.Response( + 202, json={"job_id": "ground-1", "status": "pending", "created_at": "2026-01-01T00:00:00Z"} + ) + ) + job = client.v2.ground_jobs.create(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) + assert job.job_id == "ground-1" and job.status is JobStatus.PENDING + + respx.get("https://api.ade.landing.ai/v2/ground/jobs/ground-1").mock( + return_value=httpx.Response( + 200, + json={ + "job_id": "ground-1", + "status": "completed", + "created_at": "2026-01-01T00:00:00Z", + "completed_at": "2026-01-01T00:00:09Z", + "result": GROUND_BODY, + }, + ) + ) + done = client.v2.ground_jobs.get("ground-1") + assert done.status is JobStatus.COMPLETED + assert isinstance(done.result, V2GroundResult) + assert done.result.metadata.job_id == "ground-1" + + +@respx.mock +def test_ground_job_create_sends_output_save_url() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/ground/jobs").mock( + return_value=httpx.Response(202, json={"job_id": "ground-2", "status": "pending"}) + ) + client.v2.ground_jobs.create( + extraction_metadata=EXTRACTION_METADATA, + structure=STRUCTURE, + output_save_url="https://example.com/put", + ) + req = json.loads(route.calls.last.request.content) + assert req["output_save_url"] == "https://example.com/put" + + +@respx.mock +def test_ground_job_get_failed_maps_error_object() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.get("https://api.ade.landing.ai/v2/ground/jobs/g2").mock( + return_value=httpx.Response( + 200, json={"job_id": "g2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} + ) + ) + job = client.v2.ground_jobs.get("g2") + assert job.status is JobStatus.FAILED and job.error is not None and job.error.code == "internal_error" + + +@respx.mock +def test_ground_job_get_empty_job_id_raises() -> None: + client = LandingAIADE(apikey=APIKEY) + with pytest.raises(ValueError): + client.v2.ground_jobs.get("") + + +@respx.mock +def test_ground_job_list_carries_envelope() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.get("https://api.ade.landing.ai/v2/ground/jobs").mock( + return_value=httpx.Response( + 200, + json={ + "jobs": [{"job_id": "ground-1", "status": "completed"}], + "page": 0, + "page_size": 10, + "has_more": True, + }, + ) + ) + jobs = client.v2.ground_jobs.list() + assert len(jobs) == 1 and jobs[0].job_id == "ground-1" and jobs[0].status is JobStatus.COMPLETED + assert jobs.has_more is True + assert jobs.page == 0 + assert jobs.page_size == 10 + + +@respx.mock +def test_ground_accepts_pydantic_structure() -> None: + # `structure` may be passed as a pydantic model (e.g. a parse response's + # `.structure`); it is coerced to a dict on the wire. + from landingai_ade.types.v2 import V2ParseStructure + + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/ground").mock(return_value=httpx.Response(200, json=GROUND_BODY)) + client.v2.ground(extraction_metadata=EXTRACTION_METADATA, structure=V2ParseStructure()) + req = json.loads(route.calls.last.request.content) + assert req["structure"]["type"] == "document" diff --git a/tests/contract/test_v2_smoke.py b/tests/contract/test_v2_smoke.py index 3a6dd60..50664c6 100644 --- a/tests/contract/test_v2_smoke.py +++ b/tests/contract/test_v2_smoke.py @@ -8,7 +8,7 @@ from pydantic import Field, BaseModel from landingai_ade import LandingAIADE -from landingai_ade.types.v2 import JobStatus, V2ExtractResult, V2ParseResponse +from landingai_ade.types.v2 import JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse pytestmark = pytest.mark.contract @@ -81,6 +81,34 @@ def test_parse_sync_inline_grounding_and_metadata(staging_client: LandingAIADE) assert resp.metadata.output_markdown_chars is not None +def test_ground_sync(staging_client: LandingAIADE) -> None: + # Ground is a stateless join: parse the doc, extract against it, then ground + # the extraction back onto the parse structure the markdown came from. + parsed = staging_client.v2.parse(document=Path(__file__).parent / "sample.pdf") + assert parsed.structure is not None + extracted = staging_client.v2.extract(schema=RevenueSchema, markdown=parsed.markdown or "") + grounded = staging_client.v2.ground( + extraction_metadata=extracted.extraction_metadata, + structure=parsed.structure, + ) + assert isinstance(grounded, V2GroundResult) + assert isinstance(grounded.grounding, dict) + assert grounded.metadata.job_id + + +def test_ground_jobs(staging_client: LandingAIADE) -> None: + parsed = staging_client.v2.parse(document=Path(__file__).parent / "sample.pdf") + assert parsed.structure is not None + extracted = staging_client.v2.extract(schema=RevenueSchema, markdown=parsed.markdown or "") + job = staging_client.v2.ground_jobs.create( + extraction_metadata=extracted.extraction_metadata, + structure=parsed.structure, + ) + done = staging_client.v2.ground_jobs.wait(job.job_id, timeout=300) + assert done.status is JobStatus.COMPLETED + assert isinstance(done.result, V2GroundResult) + + def test_parse_jobs(staging_client: LandingAIADE) -> None: pdf = Path(__file__).parent / "sample.pdf" job = staging_client.v2.parse_jobs.create(document=pdf) diff --git a/tests/test_v2_normalize.py b/tests/test_v2_normalize.py index be8c673..dce3c88 100644 --- a/tests/test_v2_normalize.py +++ b/tests/test_v2_normalize.py @@ -4,8 +4,12 @@ from typing import Any, Dict from datetime import datetime, timezone -from landingai_ade.types.v2 import JobStatus, V2ExtractResult, V2ParseResponse -from landingai_ade.resources.v2._normalize import normalize_parse_job, normalize_extract_job +from landingai_ade.types.v2 import JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse +from landingai_ade.resources.v2._normalize import ( + normalize_parse_job, + normalize_ground_job, + normalize_extract_job, +) def test_normalize_parse_job_epoch_and_data() -> None: @@ -139,3 +143,36 @@ def test_normalize_extract_job_unknown_status_defaults_to_pending() -> None: job = normalize_extract_job(raw) assert job.status is JobStatus.PENDING assert job.raw["status"] == "some_brand_new_status" + + +def test_normalize_ground_job_iso_and_result() -> None: + raw: Dict[str, Any] = { + "job_id": "ground-1", + "status": "completed", + "created_at": "2026-01-02T03:04:05Z", + "completed_at": "2026-01-02T03:04:09Z", + "result": { + "grounding": {"invoice_number": []}, + "metadata": {"job_id": "ground-1", "duration_ms": 10}, + }, + } + job = normalize_ground_job(raw) + assert job.status is JobStatus.COMPLETED + assert job.created_at is not None and job.created_at.year == 2026 + assert isinstance(job.result, V2GroundResult) + assert job.result.metadata.job_id == "ground-1" + + +def test_normalize_ground_job_error_object() -> None: + raw = {"job_id": "g2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} + job = normalize_ground_job(raw) + assert job.status is JobStatus.FAILED + assert job.error is not None and job.error.code == "internal_error" + + +def test_normalize_ground_job_minimal_create_envelope_defaults_to_pending() -> None: + raw = {"job_id": "ground-x"} + job = normalize_ground_job(raw) + assert job.job_id == "ground-x" + assert job.status is JobStatus.PENDING + assert job.result is None diff --git a/tests/test_v2_types.py b/tests/test_v2_types.py index 8898a4f..7140817 100644 --- a/tests/test_v2_types.py +++ b/tests/test_v2_types.py @@ -7,6 +7,7 @@ Job, JobError, JobStatus, + V2GroundResult, V2ExtractResult, V2ParseResponse, V2FileUploadResponse, @@ -212,5 +213,61 @@ def test_parse_response_inline_grounding_and_metadata() -> None: assert r.metadata.openapi_spec is not None +def test_extract_result_metadata_char_counts_warnings_and_schema_violation() -> None: + # The char counters moved onto `metadata` (from `billing`) upstream, and + # `schema_violation_error` / `warnings` were added to the result. + r = V2ExtractResult( + extraction={}, + extraction_metadata={}, + markdown="d", + metadata={ # type: ignore[arg-type] + "job_id": "e1", + "model_version": "dpt-3", + "duration_ms": 5, + "input_markdown_chars": 100, + "output_extraction_chars": 20, + }, + schema_violation_error="field 'foo' skipped", + warnings=[{"code": "partial", "message": "heads up"}], + ) + assert r.metadata.input_markdown_chars == 100 + assert r.metadata.output_extraction_chars == 20 + assert r.schema_violation_error == "field 'foo' skipped" + assert r.warnings is not None and r.warnings[0]["code"] == "partial" + + +def test_ground_result_builds_from_dicts() -> None: + r = V2GroundResult( + grounding={ + "invoice_number": [ + { + "block_id": "text-1", + "type": "text", + "grounding": {"page": 1, "range": {"start": 13, "end": 31}}, + } + ] + }, + metadata={ # type: ignore[arg-type] + "job_id": "ground-1", + "duration_ms": 5, + "openapi_spec": "https://api.example/openapi.json", + "billing": {"service_tier": "priority", "total_credits": 2.5}, + }, + ) + assert r.metadata.job_id == "ground-1" + assert r.metadata.duration_ms == 5 + assert r.metadata.billing is not None and r.metadata.billing.total_credits == 2.5 + assert "invoice_number" in r.grounding + + +def test_ground_result_retains_unknown_fields() -> None: + r = V2GroundResult( + grounding={}, + metadata={"job_id": "g", "duration_ms": 1}, # type: ignore[arg-type] + surprise=1, # type: ignore[call-arg] + ) + assert r.to_dict()["surprise"] == 1 + + def test_file_upload_response() -> None: assert V2FileUploadResponse(file_ref="abc").file_ref == "abc"