Skip to content

Harden runtime replay safety, provenance, serialization, and CI - #77

Merged
sauravsingla merged 11 commits into
mainfrom
hardening/replay-provenance-quality
Sep 25, 2026
Merged

sauravsingla merged 11 commits into
mainfrom
hardening/replay-provenance-quality

Conversation

@sauravsingla

Copy link
Copy Markdown
Owner

Summary

This hardening pass addresses four runtime and repository-quality areas together:

  • add per-run replay/idempotency protection after schema validation and authorization
  • make runtime-owned tool identity authoritative over executor-returned identity fields
  • serialize structured tool output as canonical JSON
  • strengthen central CI quality/security gates with targeted Ruff bug-pattern checks, broader mypy coverage, Bandit scanning, and reproducible hardening tests

Runtime behavior

  • ToolSpec.idempotency supports auto, none, call_id, and arguments while preserving existing positional argument ordering.
  • auto uses argument-signature replay protection for high/critical-risk tools and call-ID replay protection for standard tools.
  • only successful executions enter the per-run replay ledger; failed executions remain eligible for bounded recovery.
  • replay checks happen only after the existing schema-validation and authorization gates.
  • executor identity fields can no longer override the tool/call identity that actually passed runtime authorization; mismatches are retained only as audit metadata.
  • replay events are recorded in both telemetry and run provenance.

Structured output

ToolResult.model_content() now emits deterministic JSON for mappings/lists/tuples/scalars that are structured values, including stable key ordering and compact separators.

Quality/security gates

  • add/pin Bandit in development/CI tooling
  • add targeted Ruff bug-pattern checks on security-sensitive runtime modules
  • broaden mypy coverage to hardened runtime, HTTP transport, authorization and scope filtering
  • add Bandit medium/high severity scanning for the same security-sensitive surfaces
  • include replay/provenance tests in reproducibility lanes

Tests added

  • high-risk repeated side-effect call with a new model call ID is reused rather than executed twice during recovery
  • executor-returned spoofed identity cannot replace authorized runtime provenance
  • structured output is canonical JSON
  • invalid idempotency configuration fails closed

@sauravsingla
sauravsingla merged commit fdc59a7 into main Sep 25, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant