Skip to content

feat(agent-sdk): Add streaming support to Python agent SDK formats#1996

Open
gspencergoog wants to merge 11 commits into
unified-inference-formats-strategiesfrom
unified-inference-formats-streaming
Open

feat(agent-sdk): Add streaming support to Python agent SDK formats#1996
gspencergoog wants to merge 11 commits into
unified-inference-formats-strategiesfrom
unified-inference-formats-streaming

Conversation

@gspencergoog

@gspencergoog gspencergoog commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Important

Stacked PR Series:

  1. #1981 (Foundation & Experiments API)
  2. #1982 (Inference Strategies & Eval Solver)
  3. #1996 (Streaming Support) 👈 This PR

Summary

Implements stateful incremental parsing and fallback streaming capabilities on Python agent SDK strategies, enabling real-time token rendering for Express and Elemental formats.

Changes

  • Stateful Streaming Support:
    • Implemented stateful streaming parser logic on Parser subclasses to support incremental chunk processing.
    • Implemented StreamingPartConverter (under a2ui/a2a/parts.py) supporting active state tracking of the stream.
    • Created a robust plain text fallback parser strategy that suppresses partially rendered A2UI tags and parses incomplete blocks.
  • Robustness Refinements:
    • Updated tag prefix matching rules to support open tags with custom attributes (e.g. <a2ui id="main">).
    • Added robust validation type checks and descriptive logging when parsing streams.
  • Testing:
    • Added tests/test_streaming.py verifying stateful streaming parsing across various chunk segment variations.

Impact & Risks

  • None. Streaming parsing is backward-compatible and optimizes the response token flow.

Testing

Verify the changes by running all Python tests:

uv run pytest agent_sdks/python/a2ui_agent/tests/

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces stateful streaming support for converting token streams directly into A2A Parts by adding the StreamingPartConverter class, integrating json-repair to handle truncated JSON blocks, and adding comprehensive streaming tests. Feedback on these changes suggests suppressing trailing partial prefixes of the opening tag to prevent raw tag leakage when split across chunk boundaries, and optimizing validation by only running it on the final chunk to reduce CPU overhead and ensure a consistent user experience.

Comment on lines +186 to +192
except Exception:
open_prefix = self.parser.open_tag_prefix
open_idx = self._buffer.find(open_prefix)
if open_idx != -1:
response_parts = [ResponsePart(text=self._buffer[:open_idx])]
else:
response_parts = [ResponsePart(text=self._buffer)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

During streaming, if a chunk boundary splits the opening tag (e.g., is split into ), the partial tag will leak and be rendered to the user as plain text. To prevent this raw tag leakage and ensure a smooth user experience, we should suppress any trailing partial prefix of the opening tag at the end of the buffer. Additionally, we must enforce a minimum length of at least 3 characters for partial matches to prevent incorrect or overly loose matches.

Suggested change
except Exception:
open_prefix = self.parser.open_tag_prefix
open_idx = self._buffer.find(open_prefix)
if open_idx != -1:
response_parts = [ResponsePart(text=self._buffer[:open_idx])]
else:
response_parts = [ResponsePart(text=self._buffer)]
except Exception:
open_prefix = self.parser.open_tag_prefix
open_idx = self._buffer.find(open_prefix)
if open_idx != -1:
response_parts = [ResponsePart(text=self._buffer[:open_idx])]
else:
text = self._buffer
for i in range(len(open_prefix), 2, -1):
if text.endswith(open_prefix[:i]):
text = text[:-i]
break
response_parts = [ResponsePart(text=text)]
References
  1. When implementing substring or prefix matching logic, prioritize exact matches first and enforce a minimum length (e.g., at least 3 characters) for partial matches to prevent incorrect or overly loose matches.

Comment on lines +225 to +235
if self.validator:
try:
self.validator.validate(json_data)
except Exception as e:
if is_final:
logger.warning(
f"Failed to validate final A2UI response: {e}"
)
continue
else:
pass # Ignore validation errors for intermediate incomplete chunks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Running validation on every intermediate chunk during streaming is highly inefficient and can cause significant CPU overhead and latency spikes, especially since any validation errors are silently ignored anyway. Additionally, discarding the final UI part on validation failure (via continue) while having displayed intermediate versions of it during streaming creates an inconsistent and jarring user experience.

We should only run the validator when is_final is True, and log the warning without discarding the part.

                if self.validator and is_final:
                    try:
                        self.validator.validate(json_data)
                    except Exception as e:
                        logger.warning(
                            f"Failed to validate final A2UI response: {e}"
                        )

@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch 2 times, most recently from 498a341 to 00f6412 Compare July 13, 2026 21:35
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch 3 times, most recently from 33fc7c2 to cc24e10 Compare July 13, 2026 21:50
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from 00f6412 to fe5a35e Compare July 13, 2026 21:50
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from cc24e10 to b822a1c Compare July 13, 2026 21:56
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch 2 times, most recently from 15f4580 to 0910b7d Compare July 13, 2026 22:02
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 9f63888 to 66214f1 Compare July 13, 2026 22:12
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from 0910b7d to 4887d5c Compare July 13, 2026 22:12
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 66214f1 to 46d2441 Compare July 13, 2026 22:14
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from 4887d5c to c3da128 Compare July 13, 2026 22:14
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch 6 times, most recently from 5c340df to 556b974 Compare July 14, 2026 21:36
- Created abstract `InferenceFormat` base class and `Parser.compile` contracts.
- Relocated core transport strategy/parser/streaming files to new package structure under `a2ui.inference_formats.transport`.
- Overwrote legacy strategy and schema paths with deprecation redirects pointing to the new structure.
- Removed legacy `A2uiSchemaManager` references and renamed variables/instances to `transport_format` across all SDK tests, evaluation strategies, and sample agents.
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 556b974 to 280f9a9 Compare July 14, 2026 21:41
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 280f9a9 to 3f68fe4 Compare July 14, 2026 22:58
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from c3da128 to d0c5acc Compare July 14, 2026 22:58
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from d0c5acc to c2a9b09 Compare July 14, 2026 23:16
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 0615ce1 to 954b01e Compare July 14, 2026 23:21
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from c2a9b09 to 7cbc205 Compare July 14, 2026 23:21
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from 954b01e to d246623 Compare July 14, 2026 23:24
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from 7cbc205 to 8f263af Compare July 14, 2026 23:24
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch from d246623 to 2a8a55f Compare July 14, 2026 23:28
@gspencergoog gspencergoog force-pushed the unified-inference-formats-streaming branch from 8f263af to 3f5c74f Compare July 14, 2026 23:28
@gspencergoog gspencergoog force-pushed the unified-inference-formats-strategies branch 5 times, most recently from 0bbda91 to d495ff0 Compare July 15, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant