feat(formats): implement Express and Elemental `InferenceFormat`s by gspencergoog · Pull Request #1982 · a2ui-project/a2ui

gspencergoog · 2026-07-10T01:59:31Z

Summary

Refactors the structured inference SDK API and evaluation harness in Python to unify the handling of different inference formats (standard JSON transport, experimental Express DSL, and experimental Elemental HTML). It does this by restructuring the formats as polymorphic implementations of the InferenceFormat interface and using these abstractions to consolidate system prompt generation and response parsing logic.

Changes

SDK Package & Interface Unification:
- Restructured experimental Express and Elemental formats under agent_sdks/python/a2ui_agent/src/a2ui/inference_formats/experimental/.
- Defined polymorphic methods on the base InferenceFormat class (decompile, transform_examples, format_description, catalog_description) and implemented them across TransportFormat, ElementalFormat, and ExpressFormat.
- Updated prompt generators (ExpressPromptGenerator, ElementalPromptGenerator) to align with the new structure.
Evaluation Solvers Consolidation:
- Replaced the separate format-specific solvers (direct.py, express.py, elemental.py) under eval/a2ui_eval/strategies/ with a unified format.py solver.
- The format solver dynamically loads the correct format implementation (TransportFormat, ExpressFormat, or ElementalFormat) based on config, allowing polymorphic prompt formatting and output compilation/validation.
Robustness & General Fixes:
- Fixed case-insensitive action property resolution in ElementalCompiler when multiple action properties are present.
- Ensured prompt generator helper/catalog/decompiler are initialized in catalog_description when called prior to generate().
- Moved regular expression compilation in ElementalParser to the module level for efficiency.
- Cleaned up redundant imports and optimized invariant dataset metadata accesses in eval/ to fail fast.
Tests & Conformance:
- Renamed the conformance suite schema_manager.yaml to inference_format.yaml.
- Updated all Python SDK test suites and conformance tests to reflect the new layout and verified all 621 tests pass.

Impact & Risks

This is a breaking refactoring for internal SDK layout, but encapsulates formatting details cleanly behind the unified InferenceFormat API and preserves external integration endpoints.

Testing

Verify the changes by running all SDK and evaluation unit tests:

uv run pytest

…r API style(express): remove dead empty TYPE_CHECKING block in express prompt_generator.py feat(sdk): remove remaining A2UI_EXPRESS_ENABLED environment variable references feat(sdk): support complete flag on has_format_content for Express and Elemental feat(sdk): implement has_format_content for Express and Elemental parsers refactor(sdk): rename prompts directory to prompt (singular) refactor(sdk): hide Express and Elemental prompt generators from package exports refactor(sdk): expose prompt_generator on Express/Elemental formats and move to prompts/generator.py Refactor the Express and Elemental inference strategies to the new configuration.

…plate duplication - Promotes decompile, transform_examples, format_description, and catalog_description as abstract methods on the InferenceFormat base class. - Standardizes parameter signatures for format_description and catalog_description across Express, Elemental, and Transport formats. - Removes duplicated prompt templates and translation/compilation logic from ExpressPromptGenerator and ElementalPromptGenerator, delegating directly to the format classes. - Standardizes backwards-compatible prompt generator helpers to fallback to default format instances if initialized with raw catalog models. - Resolves all Python SDK unit tests and Kotlin conformance tests.

… clean up dataset invariant description

…e prompt rules - Remove deprecated parse_elemental_response and parse_express_response wrapper functions. - Update all integration and compiler tests to instantiate and call ElementalParser and ExpressParser directly. - Move format-specific rules (constants) and transform_examples helper methods into ElementalPromptGenerator and ExpressPromptGenerator. - Remove format_description from InferenceFormat base class and subclasses. - Retain catalog_description on the strategy classes (ElementalFormat, ExpressFormat, TransportFormat) as requested.

… error retry context - Create custom A2uiCompilationError exception carrying error message, line, column, raw content, and partial results. - Add is_final field to ResponsePart and document all fields. - Refactor ExpressParser and ElementalParser to be stateless by tracking block truncation metadata on ResponsePart objects. - Propagate is_final metadata through compile method signatures to prevent stateful tracking on parser instances. - Wrap compiler exceptions in A2uiCompilationError and attach the preceding parsed parts. - Fix greedy tag parsing in ElementalParser's block detection. - Add comprehensive integration tests for ElementalParser.

…states - Create a shared generic state-machine BlockLexer utilizing a LexerState Enum to scan LLM responses. - Correctly handle string literals (including triple quotes), backslash escapes, and code comments to prevent premature tag detection when "</a2ui>" appears inside them. - Support open tag matching with attributes (e.g. `<a2ui id="main">`). - Handle global Markdown backticks wrapper cleaning globally inside BlockLexer for all inference formats. - Refactor ExpressParser and ElementalParser to use BlockLexer for unwrapping. - Add comprehensive unit tests validating BlockLexer.

…ecompiler

… schema helpers

…ag attributes loss - Implement preserve_enclosing_tags option in BlockLexer and configure it in ElementalParser to prevent losing <a2ui> start tag attributes (like id / surfaceId). - Remove circular state mutation/double-dispatch of helper/decompiler on prompt_gen in format.catalog_description. - Override __str__ on A2uiCompilationError to output formatted error locations (line, column) and help messages. - Modernize type annotations using Python 3.10 built-in generics and PEP 604 pipe unions. - Reformat modified files with pyink.

…compiler sentinel wrapping - Refactor A2UI Elemental root component tag from <a2ui> to <body>. - Unify sentinel wrapping under <a2ui>...</a2ui> across all formats (Express and Elemental). - Update ExpressDecompiler and ElementalDecompiler to return raw unwrapped DSL/HTML strings inside decompile(). - Delegate sentinel wrapping strictly to wrap_decompiled_blocks() inside decompiler classes. - Update BlockLexer to behave as a clean tag-stripping tokenizer. - Auto-append trailing </body> in ElementalParser if compiles a truncated/unclosed creation block.

…o PromptGenerator - Remove catalog_description from InferenceFormat base interface and all format subclasses. - Add catalog_description directly to PromptGenerator subclasses (Express, Elemental, Transport). - Update prompt generator internal callers to call catalog_description locally. - Move format catalog description unit tests into prompt generator tests, and update target references.

- Declare and annotate local/instance variables in BlockLexer and TransportPromptGenerator. - Refactor _clean_markdown to always return a string, updating related test assertions. - Resolve Liskov substitution violation in A2uiTemplateManager.generate_system_prompt. - Implement stub properties for abstract InferenceFormat methods inside A2uiTemplateManager. - Typecast TypedDict dictionary queries inside TransportFormat catalog resolution.

…actoring - Update mock model output payload in test_a2ui_elemental_solvers to use <body> root tag inside <a2ui> sentinels, reflecting A2UI Elemental refactoring contract.

- Make experiments parameter conditional on version == "1.0" in eval format strategy resolver. - Support resolving surface ID from both a2ui and body tags in completion solver regex. - Auto-close unclosed a2ui tags during streaming compilation in ElementalParser. - Ensure prompt generator delegates example decompilation using its pruned catalog decompiler if available. - Call _ensure_catalog before instantiating prompt generator in ExpressFormat property. - Fix prefix check for action properties in ElementalDecompiler to look for "on-".

…s and parser classes

…tants

github-project-automation Bot added this to A2UI Jul 10, 2026

github-project-automation Bot moved this to Todo in A2UI Jul 10, 2026

gspencergoog force-pushed the unified-inference-formats-strategies branch 5 times, most recently from 34a0c18 to 0101648 Compare July 10, 2026 02:38

a2ui-project deleted a comment from gemini-code-assist Bot Jul 10, 2026

This was referenced Jul 10, 2026

refactor(formats): introduce InferenceFormat base API #1981

Merged

refactor(eval): unify evaluation strategy solvers using generic InferenceFormat solver #1983

Merged

This comment was marked as resolved.

Sign in to view

gspencergoog requested a review from jacobsimionato July 10, 2026 03:18

gspencergoog force-pushed the unified-inference-formats-strategies branch 3 times, most recently from 6238957 to 82bd501 Compare July 10, 2026 16:56

gspencergoog force-pushed the unified-inference-formats-foundation branch from 89a547a to 1134c08 Compare July 10, 2026 16:59

gspencergoog force-pushed the unified-inference-formats-strategies branch 10 times, most recently from 5266cef to 77a52dd Compare July 13, 2026 21:12

gspencergoog mentioned this pull request Jul 13, 2026

feat(agent-sdk): Add streaming support to Python agent SDK formats #1996

Open

gspencergoog force-pushed the unified-inference-formats-foundation branch from dd30576 to 914f571 Compare July 13, 2026 21:44

gspencergoog force-pushed the unified-inference-formats-strategies branch 2 times, most recently from c0e7429 to 33fc7c2 Compare July 13, 2026 21:49

gspencergoog force-pushed the unified-inference-formats-strategies branch from e910b3c to 090a859 Compare July 15, 2026 21:43

gspencergoog force-pushed the unified-inference-formats-strategies branch from 0bbda91 to d495ff0 Compare July 15, 2026 21:52

test(kotlin): update conformance YAML file path after rename

af90364

gspencergoog changed the title ~~feat(formats): implement Express and Elemental inference format strategies~~ feat(formats): implement Express and Elemental InferenceFormats Jul 15, 2026

This comment was marked as resolved.

Sign in to view

gspencergoog added 2 commits July 15, 2026 15:31

fix(sdk): address PR comments on elemental/express formats

1cc6a6e

gspencergoog force-pushed the unified-inference-formats-strategies branch from c7fd39d to 1cc6a6e Compare July 15, 2026 23:05

This comment was marked as resolved.

Sign in to view

gspencergoog added 2 commits July 15, 2026 16:21

refactor(sdk,eval): extract nested json block replacement helpers and…

eabf3f6

… clean up dataset invariant description

refactor(eval): rename generic strategy solver to format strategy solver

c5fc676

gspencergoog requested a review from nan-yu July 15, 2026 23:50

nan-yu reviewed Jul 16, 2026

View reviewed changes

gspencergoog added 11 commits July 16, 2026 11:12

test(python-sdk): increase test coverage to 88% and cover validator/d…

7230b32

…ecompiler

test(python-sdk): reach 90% test coverage with unified decompiler and…

0f4fc33

… schema helpers

test(eval): update A2UI Elemental test payload to match body root ref…

d3f1a4d

…actoring - Update mock model output payload in test_a2ui_elemental_solvers to use <body> root tag inside <a2ui> sentinels, reflecting A2UI Elemental refactoring contract.

Merge branch 'main' into unified-inference-formats-strategies

9b0b6cc

This comment was marked as resolved.

Sign in to view

gspencergoog added 4 commits July 16, 2026 14:52

docs(python-sdk): document public interfaces and properties of format…

81e3f58

…s and parser classes

refactor(python-sdk): pull out inference sentinel tags as schema cons…

e5049c3

…tants

Merge branch 'main' into unified-inference-formats-strategies

f58b0bb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(formats): implement Express and Elemental `InferenceFormat`s #1982

feat(formats): implement Express and Elemental `InferenceFormat`s #1982
gspencergoog wants to merge 21 commits into
mainfrom
unified-inference-formats-strategies

gspencergoog commented Jul 10, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

gspencergoog commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Impact & Risks

Testing

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gspencergoog commented Jul 10, 2026 •

edited

Loading