Skip to content

feat: extract the portable middleware layer into AgentCore - #29

Merged
zhanghanduo merged 2 commits into
mainfrom
feat/extract-middleware
Sep 4, 2026
Merged

zhanghanduo merged 2 commits into
mainfrom
feat/extract-middleware

Conversation

@zhanghanduo

Copy link
Copy Markdown
Collaborator

Moves everything on scripts/contract/closure.py's declared candidate list, plus four modules that were already product-neutral but missing from it. Releases as 0.6.0.

0.6.0 also carries 0.5.0's contents. 0.5.0 merged to main but was never tagged, so it never reached PyPI. CHANGELOG marks it **Never published.**, the way 0.2.0 is marked. Downstream only ever needs one pin bump.

What core was missing

It already declared ExecutionMiddleware and the structural PhaseMiddlewareChain in protocols.py but shipped no MiddlewareChain to run them. That runner arrives here, along with eleven portable middlewares, components/memory (which todo reads), and two new Protocols.

builtins.py deliberately stays in the product

It resolves a TaskContextStore through the registry, and after_phase hardcodes a research vocabulary — it scans the phase result for report / assertions / evidence_cards / clarified_questions. Neither belongs in a shared package, and it is absent from closure.py's list for that reason.

One real design change: persist_cost

It did a SQLAlchemy read-modify-write on the product's Task row. That is now the injected CostPersister Protocol.

This finishes work the file had already started — it took cost_sink and session_factory by injection, with a comment about keeping components/middleware/ from importing state/ directly. Splitting the two Protocols is the boundary:

  • CostSink.record — synchronous, per call, on the hot path, may be memory-only.
  • CostPersister.persist — once at completion, the only path that reaches a database.

summary is forwarded from the host's own get_summary without core inspecting its shape, so a host can evolve that shape without a core release.

Breaking for a caller passing session_factory — flagged under Consumer action in the CHANGELOG. Both seams still default to None, which keeps persist_cost a no-op on the stateless path.

A plan step that turned out to be wrong

The plan said move infra/llm/summary_prompt.py → runtime/loop/compact_prompt.py, the slot closure.py reserves with a comment. Not needed. Core already ships runtime/loop/summary_prompt.py with the same public names and strictly more: RESEARCH_COMPACTION_PROMPT, HANDOFF_COMPACTION_PROMPT, a compaction_prompt() selector, and a research prompt that preserves exact search queries verbatim so future turns don't re-run them.

The product's copy is a stale fork — same API, older text, quietly not receiving improvements. So compaction.py imports core's, and the product copy gets deleted rather than moved. closure.py's reserved path is stale and should go too.

Fixes the stricter gates surfaced

The product never ran pyright strict over these files:

  • LLMCallContext.metadata's field factory was unparametrised in an existing core file. Harmless while nothing read ctx.metadata deeply; now that several modules do, it made every consumer type-check as unknown.
  • base._is_retryable is only an alias for agent_core.retry_policy.legacy_retryable. retry and api_key_rotation now import the public name instead of reaching across modules for a private.
  • getattr(response, "usage", {}) or {} narrowed by isinstance yields dict[Unknown, Unknown]. Those shapes are named explicitly now.
  • Bare dict parameters and list[tuple[re.Pattern, str]] parametrised.
  • [x] + messages → [x, *messages], and if cond: return True / return False → return cond. Both provably identical.

Left alone on purpose

_record_usage_aggregator's three-level cascading TypeError fallback across three different record_llm_call signatures. Collapsing it is a behavior change that deserves its own review, so it carries a targeted noqa: SIM105 with a reason and a note in the boundary doc — not a silent rewrite.

Tests

Four product files ported: rate_limit + tool_audit + TokenBucket, token_accounting, output_repair, api_key_rotation.

New test_middleware_seams_shared.py covers the three things that had no tests anywhere — retry (back-off growth, the cap, non-retryable declined without sleeping, max-retries), tracing (metadata duration wins over the wall clock, fallback markers surface, a raising backend still returns the response), and the CostPersister seam (forwards summary + observed model, no-op without either seam, swallows a failing persister, skips a sink with no get_summary).

test_middleware.py stays in the product — it exercises builtins and needs state.event_store.sqlite and scheduling.process_manager.

Verification

  • ruff check agent_core tests scripts → clean
  • pyright agent_core (strict) → 0 errors
  • pytest -q → 1378 passed (1302 + 76)
  • uv build + twine check → PASSED
  • 0.6.0 wheel in a clean 3.12 venv: all 160 submodules import, new surface reachable
  • check_version_bump.py --base origin/main → Published code changed and version increased 0.5.0 -> 0.6.0.

Follow-up

ApodexHarness gets one PR: shims for both this and 0.5.0's cycle/verifier, the CostPersister implementation wired in at workflows/default_research/runtime.py, deletion of the stale infra/llm/summary_prompt.py, and pin ==0.6.0.

zhanghanduo and others added 2 commits September 4, 2026 14:34
Moves everything on ``scripts/contract/closure.py``'s declared candidate list
plus four modules that were already product-neutral but missing from it.
Releases as 0.6.0, which also carries 0.5.0's contents -- that version merged to
main but was never tagged, so it never reached PyPI. CHANGELOG marks it the way
0.2.0 was marked.

Core gains the phase/tool runner it was missing: it already declared
``ExecutionMiddleware`` and the structural ``PhaseMiddlewareChain`` in
``protocols.py`` but shipped no ``MiddlewareChain`` to run them. Plus eleven
portable middlewares, ``components/memory`` (which ``todo`` reads), and two new
Protocols.

``builtins.py`` deliberately stays in the product: it resolves a
``TaskContextStore`` through the registry and hardcodes a research vocabulary
(``report`` / ``assertions`` / ``evidence_cards`` / ``clarified_questions``) when
summarising a phase result. Neither belongs here.

The one real design change is ``TokenAccountingMiddleware.persist_cost``, which
did a SQLAlchemy read-modify-write on the product's ``Task`` row. That is now the
injected ``CostPersister`` protocol, and it finishes work the file had already
started -- it took ``cost_sink`` and ``session_factory`` by injection with a
comment about keeping middleware from importing ``state/`` directly. Splitting
``CostSink`` from ``CostPersister`` is the boundary: ``record`` is synchronous and
runs per call and may be memory-only, ``persist`` runs once and is the only path
that reaches a database. ``summary`` is forwarded from the host's own
``get_summary`` without core inspecting its shape, so the host can evolve it
without a core release. This is a breaking signature change for a caller passing
``session_factory``; the CHANGELOG says so under Consumer action.

``infra/llm/summary_prompt.py`` turned out NOT to need moving, contrary to the
plan and to closure.py's reserved ``core/runtime/loop/compact_prompt.py`` slot.
Core already ships ``runtime/loop/summary_prompt.py`` with the same public names
and strictly more: ``RESEARCH_COMPACTION_PROMPT``, ``HANDOFF_COMPACTION_PROMPT``,
a ``compaction_prompt()`` selector, and a research prompt that preserves exact
search queries. The product's copy is a stale fork. So ``compaction.py`` now
imports core's, and the product copy should be deleted rather than moved.

Fixes found by AgentCore's stricter gates, which the product's config never ran
over these files:

- ``LLMCallContext.metadata``'s factory was unparametrised in an existing core
  file. Harmless until something read ``ctx.metadata`` deeply; now that several
  modules do, it made every consumer type-check as unknown.
- ``base._is_retryable`` is only an alias for
  ``agent_core.retry_policy.legacy_retryable``; ``retry`` and
  ``api_key_rotation`` now import the public name instead of reaching for a
  private across modules.
- ``getattr(response, "usage", {}) or {}`` narrowed by ``isinstance`` yields
  ``dict[Unknown, Unknown]``; those shapes are named explicitly now.
- Bare ``dict`` parameters and ``list[tuple[re.Pattern, str]]`` are parametrised.
- ``[x] + messages`` and an ``if cond: return True / return False`` collapsed --
  both provably identical.

Left alone on purpose: ``_record_usage_aggregator``'s three-level cascading
``TypeError`` fallback across three kwarg signatures. Collapsing it is a behavior
change that deserves its own review, so it carries a targeted ``noqa`` and a
note in the boundary doc rather than a silent rewrite.

Tests: four product files ported (``rate_limit`` + ``tool_audit`` +
``TokenBucket``, ``token_accounting``, ``output_repair``, ``api_key_rotation``),
and a new ``test_middleware_seams_shared.py`` covering the three things that had
no tests anywhere -- ``retry``, ``tracing``, and the ``CostPersister`` seam
including its no-op, swallow-failure and missing-``get_summary`` paths.
``test_middleware.py`` stays in the product: it exercises ``builtins`` and needs
``state.event_store.sqlite`` and ``scheduling.process_manager``.

Verified: ruff clean over agent_core/tests/scripts, pyright strict 0 errors,
pytest 1378 passed (1302 + 76), uv build + twine check PASSED, and the 0.6.0
wheel installed into a clean 3.12 venv imports all 160 submodules with the new
surface reachable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@zhanghanduo
zhanghanduo merged commit b844072 into main Sep 4, 2026
5 checks passed
@zhanghanduo
zhanghanduo deleted the feat/extract-middleware branch September 4, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant