feat(agent-sdk): Add streaming support to Python agent SDK formats#1996
feat(agent-sdk): Add streaming support to Python agent SDK formats#1996gspencergoog wants to merge 11 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces stateful streaming support for converting token streams directly into A2A Parts by adding the StreamingPartConverter class, integrating json-repair to handle truncated JSON blocks, and adding comprehensive streaming tests. Feedback on these changes suggests suppressing trailing partial prefixes of the opening tag to prevent raw tag leakage when split across chunk boundaries, and optimizing validation by only running it on the final chunk to reduce CPU overhead and ensure a consistent user experience.
| except Exception: | ||
| open_prefix = self.parser.open_tag_prefix | ||
| open_idx = self._buffer.find(open_prefix) | ||
| if open_idx != -1: | ||
| response_parts = [ResponsePart(text=self._buffer[:open_idx])] | ||
| else: | ||
| response_parts = [ResponsePart(text=self._buffer)] |
There was a problem hiding this comment.
During streaming, if a chunk boundary splits the opening tag (e.g., is split into ), the partial tag will leak and be rendered to the user as plain text. To prevent this raw tag leakage and ensure a smooth user experience, we should suppress any trailing partial prefix of the opening tag at the end of the buffer. Additionally, we must enforce a minimum length of at least 3 characters for partial matches to prevent incorrect or overly loose matches.
| except Exception: | |
| open_prefix = self.parser.open_tag_prefix | |
| open_idx = self._buffer.find(open_prefix) | |
| if open_idx != -1: | |
| response_parts = [ResponsePart(text=self._buffer[:open_idx])] | |
| else: | |
| response_parts = [ResponsePart(text=self._buffer)] | |
| except Exception: | |
| open_prefix = self.parser.open_tag_prefix | |
| open_idx = self._buffer.find(open_prefix) | |
| if open_idx != -1: | |
| response_parts = [ResponsePart(text=self._buffer[:open_idx])] | |
| else: | |
| text = self._buffer | |
| for i in range(len(open_prefix), 2, -1): | |
| if text.endswith(open_prefix[:i]): | |
| text = text[:-i] | |
| break | |
| response_parts = [ResponsePart(text=text)] |
References
- When implementing substring or prefix matching logic, prioritize exact matches first and enforce a minimum length (e.g., at least 3 characters) for partial matches to prevent incorrect or overly loose matches.
| if self.validator: | ||
| try: | ||
| self.validator.validate(json_data) | ||
| except Exception as e: | ||
| if is_final: | ||
| logger.warning( | ||
| f"Failed to validate final A2UI response: {e}" | ||
| ) | ||
| continue | ||
| else: | ||
| pass # Ignore validation errors for intermediate incomplete chunks |
There was a problem hiding this comment.
Running validation on every intermediate chunk during streaming is highly inefficient and can cause significant CPU overhead and latency spikes, especially since any validation errors are silently ignored anyway. Additionally, discarding the final UI part on validation failure (via continue) while having displayed intermediate versions of it during streaming creates an inconsistent and jarring user experience.
We should only run the validator when is_final is True, and log the warning without discarding the part.
if self.validator and is_final:
try:
self.validator.validate(json_data)
except Exception as e:
logger.warning(
f"Failed to validate final A2UI response: {e}"
)498a341 to
00f6412
Compare
33fc7c2 to
cc24e10
Compare
00f6412 to
fe5a35e
Compare
cc24e10 to
b822a1c
Compare
15f4580 to
0910b7d
Compare
9f63888 to
66214f1
Compare
0910b7d to
4887d5c
Compare
66214f1 to
46d2441
Compare
4887d5c to
c3da128
Compare
5c340df to
556b974
Compare
- Created abstract `InferenceFormat` base class and `Parser.compile` contracts. - Relocated core transport strategy/parser/streaming files to new package structure under `a2ui.inference_formats.transport`. - Overwrote legacy strategy and schema paths with deprecation redirects pointing to the new structure. - Removed legacy `A2uiSchemaManager` references and renamed variables/instances to `transport_format` across all SDK tests, evaluation strategies, and sample agents.
556b974 to
280f9a9
Compare
280f9a9 to
3f68fe4
Compare
c3da128 to
d0c5acc
Compare
d0c5acc to
c2a9b09
Compare
0615ce1 to
954b01e
Compare
c2a9b09 to
7cbc205
Compare
954b01e to
d246623
Compare
7cbc205 to
8f263af
Compare
…te parse_response_to_parts
…nd move to prompts/generator.py
d246623 to
2a8a55f
Compare
8f263af to
3f5c74f
Compare
0bbda91 to
d495ff0
Compare
Important
Stacked PR Series:
#1996(Streaming Support) 👈 This PRSummary
Implements stateful incremental parsing and fallback streaming capabilities on Python agent SDK strategies, enabling real-time token rendering for Express and Elemental formats.
Changes
Parsersubclasses to support incremental chunk processing.StreamingPartConverter(undera2ui/a2a/parts.py) supporting active state tracking of the stream.<a2ui id="main">).tests/test_streaming.pyverifying stateful streaming parsing across various chunk segment variations.Impact & Risks
Testing
Verify the changes by running all Python tests: