diff --git a/docs/acceptance.md b/docs/acceptance.md index 1b5cc60c..e4d41e28 100644 --- a/docs/acceptance.md +++ b/docs/acceptance.md @@ -42,7 +42,7 @@ acceptance flow writes the user TUI state file. In sandboxed environments, run them with permission to bind localhost ports and write the TeaAgent state directory. -**Current acceptance test count: `673 passed`** (pytest-collected guard target) +**Current acceptance test count: `674 passed`** (pytest-collected guard target) Keep historical acceptance-count snapshots in dated analysis or roadmap docs. This file only owns the live guard target. diff --git a/docs/backlog-priority.md b/docs/backlog-priority.md index 9a0143ce..a3415d4e 100644 --- a/docs/backlog-priority.md +++ b/docs/backlog-priority.md @@ -513,7 +513,7 @@ Then manually refresh [scripts/refresh_agent_readme_survey.md](../scripts/refres Items below were found in older plan and review documents and are not tracked in this file or `roadmap-status.md`. They carry **no priority, no schedule, and no shipped/beta status**. New work enters only through the DR-006 gate (see the Backlog Provenance rules above). Each row was checked against its cited source line on 2026-10-26. -**Triage pass (2026-10-26):** rows marked **PROPOSED** carry a verdict checked against code and tests on that date; they take effect only when the owner ratifies them. Proposed so far: 6 Close-as-done, 1 Close-as-obsolete, 5 Hold; WS2-002 and the telemetry-test row were fixed directly (a runtime-limit defect and a CI break). Three rows were tagged `governance-gap` candidates and were resolved on 2026-10-26 with tests: WS2-002 and DU-010A were real defects and are fixed; DU-016 turned out to have no bypass and is now pinned by tests. PHASE-01…06 and the 10 open DU items have no verdict yet. +**Triage pass (2026-10-26):** rows marked **PROPOSED** carry a verdict checked against code and tests on that date; they take effect only when the owner ratifies them. Proposed so far: 6 Close-as-done, 1 Close-as-obsolete, 5 Hold; WS2-002 and the telemetry-test row were fixed directly (a runtime-limit defect and a CI break). Three rows were tagged `governance-gap` candidates and were resolved on 2026-10-26 with tests: WS2-002 and DU-010A were real defects and are fixed; DU-016 turned out to have no bypass and is now pinned by tests. **Second pass (2026-10-27):** decision packets re-checked the rest against code and the friction log. DENY-OVERRIDE turned out to be already decided by the owner and was fixed (G-P1-4); DU-011 and DU-014 fall under the standing DR-006 M4 carve-out and were built (U-P1-7, U-P1-8); the WS2-002 residual was fixed with the existing cancel token (G-P1-5). PHASE-01…06 and DU-008/010/015/017/018/020 now carry PROPOSED verdicts (one close-as-superseded, eleven holds); none has friction or governance-gap evidence. | ID | Title | Source file:line | Provenance | Status | | --- | --- | --- | --- | --- | @@ -523,7 +523,7 @@ Items below were found in older plan and review documents and are not tracked in | approval_manager.py split | Split `approval_manager.py` (1205 lines); listed as still pending follow-up | docs/plans/improvement-work-items.md:100 | none | **PROPOSED Hold** — now `teaagent/approval/manager.py`, 1,473 lines and growing; its god-module exemption (`scripts/check_god_modules.py:27`) says "migration completed" with no ADR — owner to confirm the exemption is still valid | | WS1-002 | Replace raw approval-ID-first UX with readable selectors | docs/plans/system-improvement-work-directions-2026-06-06.md:66 | n/a | **PROPOSED Close-as-done** — selectors with tool/reason/path/risk/expiry in `teaagent/approval/selectors.py:22-26`; `approval approve --selector N` | | WS1-003 | Add default progress summaries for long runs | docs/plans/system-improvement-work-directions-2026-06-06.md:67 | n/a | **PROPOSED Close-as-done** — `teaagent/run_progress.py:15-20` wired into TUI and CLI; `tests/test_run_progress.py` | -| WS2-002 | Add batch-level timeout and cancellation to subagent batch execution | docs/plans/system-improvement-work-directions-2026-06-06.md:81 | `governance-gap` (runtime limit not enforced) | **Done 2026-10-26** — `teaagent/subagents/_tools.py` no longer blocks on running subagents after the batch deadline (`shutdown(wait=False, cancel_futures=True)`) and labels them "while task was running; result discarded"; `tests/test_subagent_batch_timeout_running.py` failed at 5.00s before the fix. Residual: abandoned threads keep running (no cooperative cancel hook) and the interpreter still joins them at exit | +| WS2-002 | Add batch-level timeout and cancellation to subagent batch execution | docs/plans/system-improvement-work-directions-2026-06-06.md:81 | `governance-gap` (runtime limit not enforced) | **Done 2026-10-26; residual closed 2026-10-27 (G-P1-5)** — each batch now passes its own `cancel_token` to its children and sets it at the deadline; the runner checks it before every tool dispatch, so at most one already-dispatched call can finish after the batch returns (`tests/test_subagent_batch_cancel_token.py`: 4 post-deadline calls before, 0 after). Cancellation is audited as the existing `run_failed` (`system`). The interpreter still joins worker threads at exit | | WS3-004 | Define cost state taxonomy (estimated, provider-reported, pending, unknown) | docs/plans/system-improvement-work-directions-2026-06-06.md:99 | n/a | **PROPOSED Close-as-done** — `teaagent/cost_state.py` (`derive_cost_state`, `normalize_cost_state`) used by `evidence_summary.py:353` and `ergonomics/run_summary.py:89`; `tests/test_cost_state_taxonomy.py` 6/6. Residual: no adapter reports `provider_reported` yet (`actual_cost_cents` stays None) | | WS4-002 | Expose approval queue depth and age | docs/plans/system-improvement-work-directions-2026-06-06.md:112 | n/a | **PROPOSED Close-as-done** — queue age/risk/expiry and `queue_depth` in `teaagent/approval/selectors.py:225-238`, consumed by CLI and TUI | | WDA-006 | Decide and document whether consensus validation gates destructive actions behind the approval queue or is explicitly deferred (ADR with expiry) | docs/plans/work-direction-decomposition-2026-06-10.md:41 | n/a | **PROPOSED Close-as-done** — ADR-0029 closed 2026-07-22 (delete/quarantine); destructive actions use the approval queue (ADR-0022); `teaagent/consensus/` is ADR-0043 row 5, not this item | @@ -531,20 +531,20 @@ Items below were found in older plan and review documents and are not tracked in | Remote HTTP approval | Remote approval orchestration over HTTP (backend stub exists; implementation deferred) | docs/plans/subagent-swarm-orchestration-unification-2026-06-06.md:102 | `legacy-competitive` (remote multi-agent non-goal) | **PROPOSED Close-as-obsolete** — premise stale: opt-in HTTP backend exists (`teaagent/coordination/approval_backend.py:174-210`, `approval_http_client.py`, `approval_http_server.py`, `tests/test_http_remote_approval_backend.py`; default backend is `file`). The owner should place this surface at the ADR-0043 2026-12-09 review | | Foreign-hook fail-open friction F10 / SRV-PI-07 veto | Host hook fail-open on failure; a TeaAgent PreToolUse veto over foreign tool calls is not built | docs/reviews/architecture-rethink-2026-10-23.md:76 (SRV-PI-07); :123 (F10) | `friction-driven` (F10, `work-log/operator-friction-log.md:195`) | **PROPOSED Hold (keeps owner decision)** — veto deliberately not built; a build needs a dated owner request (`reviews/foreign-hook-safer-path-2026-10-23.md:7,47`) | | Telemetry test pins a private OTel attribute | `tests/test_telemetry.py:376` reads `provider._sdk_config.metric_readers`, removed in opentelemetry-sdk 1.44; passes only at the locked 1.41.1, so the next lock bump breaks it | `tests/test_telemetry.py:364-380`; found in the 2026-10-26 full-suite run | CI-health (blocked PR CI once opentelemetry-sdk ≥1.44 was installed) | **Done 2026-10-26** — the test now spies on the public `PeriodicExportingMetricReader`/`MeterProvider` constructors; passes on opentelemetry-sdk 1.41.1 and 1.44.0 | -| PHASE-01 | Persistent Context — Create CLAUDE.md & Refactor AGENTS.md (13 open boxes) | `docs/plans/phase-01-persistent-context.md` | none | Untriaged — needs DR-006 gate. `CLAUDE.md` does not exist; AGENTS.md still holds the architecture rules | -| PHASE-02 | Scope Governance — Feature Necessity Audit (15 open boxes) | `docs/plans/phase-02-scope-audit.md` | none | Untriaged — needs DR-006 gate. Gating shipped Beta CLI surfaces needs an owner decision; doc paths `docs/operations/` should read `docs/ops/` | -| PHASE-03 | Competitive Analysis — Structured Devil's Advocate (16 open boxes) | `docs/plans/phase-03-competitive-analysis.md` | `legacy-competitive` candidate | Untriaged — needs DR-006 gate. Competitor analyses exist; held under DR-006 T1 unless owner-override | -| PHASE-04 | PMF Measurement Framework (24 open boxes) | `docs/plans/phase-04-pmf-framework.md` | `legacy-competitive` candidate | Untriaged — needs DR-006 gate. Baseline is mostly synthetic (0 owner-origin runs, `roadmap-status.md:15`); overlaps VND-003/AGF-003 holds | -| PHASE-05 | Founder Systematization — Reduce Single-Point-of-Failure Risk (24 open boxes) | `docs/plans/phase-05-founder-systematization.md` | none | Untriaged — needs DR-006 gate. Release/CI runbooks partly exist (`docs/governance/release-process.md`, `docs/ops/runbooks.md`) | -| PHASE-06 | Data Flywheel — Turn Usage Signals into Product Improvement (23 open boxes) | `docs/plans/phase-06-data-flywheel.md` | none | Untriaged — needs DR-006 gate. Failure analysis partly exists (`teaagent/run_store.py`, `memory/failure_card.py`); overlaps VND-003/AGF-003 holds | -| DU-008 | Command grammar doc + parser test | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:57` | none | Untriaged — needs DR-006 gate. No grammar doc or parser test found; Phase 1 release criterion | -| DU-010 | Approval/recovery copy taxonomy (provider-missing, undo-unavailable) | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:59` | none | Untriaged — needs DR-006 gate. Partial: `tests/test_guided_recovery.py:25-64` covers approval and budget only | +| PHASE-01 | Persistent Context — Create CLAUDE.md & Refactor AGENTS.md (13 open boxes) | `docs/plans/phase-01-persistent-context.md` | none | **PROPOSED Hold (2026-10-27)** — AGENTS.md already carries the architecture, tool-governance, runtime-safety and skills rules that the plan would move into `CLAUDE.md`; no friction or governance gap | +| PHASE-02 | Scope Governance — Feature Necessity Audit (15 open boxes) | `docs/plans/phase-02-scope-audit.md` | `legacy-competitive` | **PROPOSED Close-as-superseded (2026-10-27)** — feature-surface gating is the ADR-0043 quarantine disposition (review 2026-12-09); non-goals already live in `strategy/harness-first-direction-2026-06-13.md` | +| PHASE-03 | Competitive Analysis — Structured Devil's Advocate (16 open boxes) | `docs/plans/phase-03-competitive-analysis.md` | `legacy-competitive` | **PROPOSED Hold (2026-10-27)** — competitor material is hypothesis intake only under DR-006; persona and positioning are business work outside this engineering backlog | +| PHASE-04 | PMF Measurement Framework (24 open boxes) | `docs/plans/phase-04-pmf-framework.md` | `legacy-competitive` | **PROPOSED Hold (2026-10-27)** — PMF measurement is business work; run data is synthetic (0 owner-origin runs); opt-in telemetry would be a new surface; overlaps the VND-003/AGF-003 holds | +| PHASE-05 | Founder Systematization — Reduce Single-Point-of-Failure Risk (24 open boxes) | `docs/plans/phase-05-founder-systematization.md` | none | **PROPOSED Hold (2026-10-27)** — process and staffing; the release and CI runbook boxes could close as superseded by `governance/release-process.md` and `ops/runbooks.md` if the owner agrees | +| PHASE-06 | Data Flywheel — Turn Usage Signals into Product Improvement (23 open boxes) | `docs/plans/phase-06-data-flywheel.md` | none | **PROPOSED Hold (2026-10-27)** — reporting, not audit or approval correctness; RunStore misses foreign-harness work (friction F9), so it is an incomplete data source; overlaps the VND-003/AGF-003 holds | +| DU-008 | Command grammar doc + parser test | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:57` | none | **PROPOSED Hold (2026-10-27)** — confirmed: no grammar doc or parser-order test; no friction-log entry, not a governance gap | +| DU-010 | Approval/recovery copy taxonomy (provider-missing, undo-unavailable) | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:59` | none | **PROPOSED Hold (2026-10-27)** — partial, broader than first recorded: `tests/test_guided_recovery.py:25-141` covers 7 failure categories, plus provider-missing, unknown-command and nothing-to-undo copy tests; no consolidated taxonomy; no open friction entry (F7 closed) | | DU-010A | Cost-cap zero semantics | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:60` | `governance-gap` (budget) | **Done 2026-10-26** — `cli/_handlers/_agent/automation.py` maps a 0 cap to unset, so ticks run under the child `agent run` default (500 cents) instead of a zero-spend cap; `tests/test_tick_cost_cap_zero_means_unset.py` | -| DU-011 | TUI cockpit field contract (headless assertion) | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:67` | none | Untriaged — needs DR-006 gate. `tests/test_control_cockpit.py` checks the dataclass only | -| DU-014 | Long-run heartbeat with tool/cost/checkpoint fields | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:70` | none | Untriaged — needs DR-006 gate. `teaagent/heartbeat.py:13-32` tracks tick_count only | -| DU-015 | Compaction preview before `/compact` | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:71` | none | Untriaged — needs DR-006 gate. `tui/core.py:988-1011` compacts without preview | +| DU-011 | TUI cockpit field contract (headless assertion) | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:67` | `owner-override` (co-maintainer dogfood; DR-006 M4 operator-cockpit carve-out) | **Done 2026-10-27 (U-P1-7)** — `ControlCockpitState` carries branch, undo availability, last run id and active rules, rendered in the TUI cockpit; each source fails to an explicit `unknown` and is logged (`tests/test_control_cockpit.py`) | +| DU-014 | Long-run heartbeat with tool/cost/checkpoint fields | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:70` | `owner-override` (co-maintainer dogfood; DR-006 M4 background-lifecycle carve-out) | **Done 2026-10-27 (U-P1-8)** — heartbeat ticks carry `tool`, `tool_in_flight`, `elapsed_seconds`, `cost_cents` and `checkpoint` (last completed call; the runner has no next-checkpoint concept) via a provider wired in `chat_agent.py`; the liveness file still holds tick/interval only (`tests/test_heartbeat_progress_fields.py`) | +| DU-015 | Compaction preview before `/compact` | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:71` | none | **PROPOSED Hold (2026-10-27)** — confirmed: `/compact` compacts without preview (`teaagent/tui/core.py:988-1011`); no friction entry | | DU-016 | MCP unknown-tool default prompt | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:77` | `governance-gap` candidate (approval) | **Done 2026-10-26 (no bypass)** — every MCP tool is annotated destructive with remote hints ignored (`teaagent/mcp_tool_adapter.py:52-66`), so unknown tools reach `ApprovalPolicy` in prompt, workspace-write and read-only modes; pinned through `AgentRunner` by `tests/test_mcp_unknown_tool_prompt.py` (6 tests). `mcp_trust.check_unknown_tool_prompt` remains unused production code | -| DU-017 | Skill/rule/memory-source inspectability in TUI and run evidence | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:78` | none | Untriaged — needs DR-006 gate. Skills are listed; project instructions and memory sources are not | -| DU-018 | First-hour smoke script | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:79` | none | Untriaged — needs DR-006 gate. `tests/acceptance/test_first_hour_e2e_flow.py` lacks TUI chat/cost/docs steps; no `scripts/` entry | -| DU-020 | README readiness-badge discipline | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:81` | none | Untriaged — needs DR-006 gate. No README readiness gate found | -| DENY-OVERRIDE | Interactive approval can override a deny preset | `teaagent/approval/manager.py:987` raises `ToolPermissionError(POLICY_DENIED)` for a deny preset; `teaagent/runner/_governed_execution.py:314-326` catches every `ToolPermissionError` and offers interactive approval, so an approving handler runs a tool the owner denied. Found 2026-10-26 while pinning dogfood G1 | `governance-gap` candidate (approval) | **Untriaged — needs owner decision:** should an explicit deny preset be terminal, or may a person at the prompt override it? Current behavior is pinned only for allow mode and no-handler prompt mode (`tests/test_deny_preset_runtime_enforcement.py`) | +| DU-017 | Skill/rule/memory-source inspectability in TUI and run evidence | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:78` | none | **PROPOSED Hold (2026-10-27)** — partial: `teaagent guidance` lists AGENTS.md and the TUI has `/skills` and `/memory`; TUI and run evidence still do not show project-instruction or memory sources; no friction entry | +| DU-018 | First-hour smoke script | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:79` | none | **PROPOSED Hold (2026-10-27)** — confirmed: first-hour e2e covers setup/daily/edit/undo only; no TUI, cost or docs steps and no `scripts/` entry; no friction entry | +| DU-020 | README readiness-badge discipline | `docs/archive/plans/daily-driver-usefulness-master-plan-2026-06-01.md:81` | none | **PROPOSED Hold (2026-10-27)** — confirmed: README has readiness sections but no badge gate; docs-claim hygiene is gated separately (DR-006) | +| DENY-OVERRIDE | Interactive approval can override a deny preset | `teaagent/approval/manager.py:987` raises `ToolPermissionError(POLICY_DENIED)` for a deny preset; `teaagent/runner/_governed_execution.py:314-326` catches every `ToolPermissionError` and offers interactive approval, so an approving handler runs a tool the owner denied. Found 2026-10-26 while pinning dogfood G1 | `governance-gap` (approval) | **Done 2026-10-27 (G-P1-4)** — the owner had already decided "block in all modes" (`reviews/dogfood-g1-g22-2026-09-15-risk.md:22`; `cli.md:923`), so this was a defect, not an open question. `POLICY_DENIED` now goes to `record_blocked` before any interactive approval (`teaagent/runner/_governed_execution.py`); `approval revoke ` is the escape hatch (falsifier test in `tests/test_deny_preset_runtime_enforcement.py`). Protected-path codes (`WORKSPACE_WRITE_MODE`, `SYMLINK_BLOCKED`, `SKILL_WRITE_BLOCKED`) still reach the approval prompt for destructive tools — unchanged | diff --git a/docs/generated/docs-aging-dashboard.md b/docs/generated/docs-aging-dashboard.md index 54c07249..0703c1da 100644 --- a/docs/generated/docs-aging-dashboard.md +++ b/docs/generated/docs-aging-dashboard.md @@ -4,7 +4,7 @@ **Stale threshold:** 90 days since `Last reviewed` **Current-truth docs scanned:** 17 -**Needs attention (working tier only):** 7 +**Needs attention (working tier only):** 9 **Archive-tier docs (exempt from staleness):** 0 Regenerate: `python3 scripts/report_docs_aging.py` @@ -29,7 +29,7 @@ Regenerate: `python3 scripts/report_docs_aging.py` | Document | Status | Last reviewed | File mtime | Notes | | --- | --- | --- | --- | --- | -| `docs/maturity-matrix.md` | stale_by_mtime | 2026-09-16 | 2026-10-26 | Missing owner banner; File modified after last reviewed date | +| `docs/maturity-matrix.md` | stale_by_mtime | 2026-09-16 | 2026-10-27 | Missing owner banner; File modified after last reviewed date | ### project @@ -41,7 +41,19 @@ Regenerate: `python3 scripts/report_docs_aging.py` | Document | Status | Last reviewed | File mtime | Notes | | --- | --- | --- | --- | --- | -| `docs/roadmap-status.md` | stale_by_mtime | 2026-10-23 | 2026-10-26 | Missing review trigger banner; File modified after last reviewed date | +| `docs/roadmap-status.md` | stale_by_mtime | 2026-10-23 | 2026-10-27 | Missing review trigger banner; File modified after last reviewed date | + +### strategy + +| Document | Status | Last reviewed | File mtime | Notes | +| --- | --- | --- | --- | --- | +| `docs/backlog-priority.md` | stale_by_mtime | 2026-10-26 | 2026-10-27 | Missing owner banner; File modified after last reviewed date | + +### verification + +| Document | Status | Last reviewed | File mtime | Notes | +| --- | --- | --- | --- | --- | +| `docs/acceptance.md` | stale_by_mtime | 2026-09-16 | 2026-10-27 | File modified after last reviewed date | ## Review Triggers (Current-Truth Docs) @@ -67,10 +79,10 @@ Regenerate: `python3 scripts/report_docs_aging.py` ## Corpus Cost (G5 Signal) -**Total docs:** 675 (baseline 582 at diagnosis, delta +93) -**Live corpus (non-archive):** 377 +**Total docs:** 676 (baseline 582 at diagnosis, delta +94) +**Live corpus (non-archive):** 378 **Archive-tier docs (total):** 298 -**Working-tier docs unreferenced by INDEX.md:** 323 +**Working-tier docs unreferenced by INDEX.md:** 324 Dead-weight candidates (working tier, not linked from INDEX.md): @@ -94,7 +106,7 @@ Dead-weight candidates (working tier, not linked from INDEX.md): - `adr/0017-backend-adapter-interfaces.md` - `adr/0018-async-from-sync-pattern.md` - `adr/0019-phase-4-federated-swarm-consensus.md` -- ... and 303 more +- ... and 304 more ### Archive-tier growth (files added per month, last 6 months) diff --git a/docs/generated/docs-inventory.md b/docs/generated/docs-inventory.md index 939f1527..8dee9bb1 100644 --- a/docs/generated/docs-inventory.md +++ b/docs/generated/docs-inventory.md @@ -6,11 +6,11 @@ Generated by `python3 scripts/generate_docs_inventory.py`. Do not edit this file manually — regenerate instead. -**Markdown files:** 674 +**Markdown files:** 675 | Path | Tier | Bytes | SHA256 (12) | | --- | --- | ---: | --- | -| `acceptance.md` | constitution | 22281 | `eb345d401d1c` | +| `acceptance.md` | constitution | 22281 | `175a778989b1` | | `adr/0001-p0-framework.md` | working | 3225 | `d0666420a497` | | `adr/0002-p1-primitives.md` | working | 2877 | `d50c03f403a6` | | `adr/0003-p2-code-mode-sandbox.md` | working | 3153 | `594b44eb1569` | @@ -211,7 +211,7 @@ Do not edit this file manually — regenerate instead. | `archive/plans/system-transparency-engineering-plan-2026-05-31.md` | archive | 15408 | `81c7914ec72d` | | `archive/plans/ux-improvement-roadmap-2026-05-31.md` | archive | 15447 | `b77390585263` | | `audit-events.md` | working | 2376 | `b83066076fba` | -| `backlog-priority.md` | working | 55028 | `b8457396b4b2` | +| `backlog-priority.md` | working | 57277 | `545b5e977389` | | `cli.md` | working | 42072 | `a02a140f784d` | | `cloud-deployment.md` | working | 9439 | `b27aada996c9` | | `context-bus-and-federated-sync.md` | working | 1564 | `2dde6eed4bf8` | @@ -259,7 +259,7 @@ Do not edit this file manually — regenerate instead. | `error-reference.md` | working | 4897 | `067448aba6f1` | | `gateway-oauth-tenants.md` | working | 1818 | `0ff9c7df4652` | | `generated/command-snippet-inventory.md` | working | 8128 | `bf21da5dfe66` | -| `generated/docs-aging-dashboard.md` | working | 5108 | `03e9c10505fb` | +| `generated/docs-aging-dashboard.md` | working | 5570 | `6a62a1d29358` | | `generated/release-docs-evidence.md` | working | 3821 | `acee07427ed9` | | `governance-compliance.md` | constitution | 1645 | `d7665f2f7864` | | `governance/code-review-checklist.md` | working | 4379 | `f580da208c75` | @@ -318,7 +318,7 @@ Do not edit this file manually — regenerate instead. | `implementation/daily-driver-fix-log.md` | working | 978 | `474648b41793` | | `implementation/fix-log-2026-06-02.md` | archive | 8520 | `21239812e63e` | | `INDEX.md` | working | 22539 | `40f6954ac592` | -| `maturity-matrix.md` | working | 7159 | `6b992699e381` | +| `maturity-matrix.md` | working | 7159 | `2df78e17decd` | | `migration-top-level-api.md` | working | 1505 | `2ba32dedee5a` | | `model-capability-matrix.md` | working | 4654 | `4843aa15a016` | | `modules/approval_manager/api.md` | working | 3905 | `37c89490637d` | @@ -535,11 +535,12 @@ Do not edit this file manually — regenerate instead. | `retrospective/03-architecture-quality.md` | working | 27525 | `fd28fdd70665` | | `retrospective/04-ux-usability.md` | working | 26253 | `a9350bc35c4b` | | `retrospective/05-compliance-matrix.md` | working | 8285 | `cd2568b5eddb` | -| `retrospective/06-action-register.md` | working | 45379 | `148cd2acd733` | +| `retrospective/06-action-register.md` | working | 47037 | `556e9dccdc74` | | `retrospective/automation-plan.md` | working | 18716 | `0e0027c4b1c5` | | `retrospective/README.md` | working | 8718 | `d93f2a71f6b3` | | `retrospective/review-system.md` | working | 15360 | `ce42e83e5bcf` | | `retrospective/tool-capability-review.md` | working | 15681 | `20ca7506fc04` | +| `reviews/98-risk.md` | working | 5511 | `710b7796952f` | | `reviews/a-p0-2-observability-risk.md` | working | 2781 | `8a2295be6857` | | `reviews/a-p1-3-authorization-extraction-risk.md` | working | 6913 | `fe84ae18f08b` | | `reviews/a-p1-3-governed-execution-risk.md` | working | 4979 | `bcb6e5e1fb88` | @@ -585,7 +586,7 @@ Do not edit this file manually — regenerate instead. | `reviews/sec09-sec15-multisig-hardening-risk.md` | working | 11076 | `ae895567bb02` | | `reviews/security-risk-assessment-2026-06-02.md` | archive | 24112 | `4c9e2e00d001` | | `reviews/seven-control-loops-critical-questioning-2026-06-05.md` | archive | 7531 | `ae1e34b8369d` | -| `roadmap-status.md` | constitution | 113958 | `19c400a3ad1e` | +| `roadmap-status.md` | constitution | 114662 | `9df14453d98f` | | `run-evidence-and-audit-guide.md` | working | 2494 | `5ef9c11997d7` | | `security-whitepaper.md` | working | 9690 | `87fc3c31c4a3` | | `security/approval-abuse-cases-2026-06-02.md` | archive | 1281 | `4c43296d1c66` | diff --git a/docs/maturity-matrix.md b/docs/maturity-matrix.md index 8e88cc2a..b56ddc9f 100644 --- a/docs/maturity-matrix.md +++ b/docs/maturity-matrix.md @@ -1,6 +1,6 @@ # TeaAgent Maturity Matrix -> **Last reviewed:** 2026-09-16 (labels unchanged; acceptance posture 131 files / 673 tests matches guard) +> **Last reviewed:** 2026-09-16 (labels unchanged; acceptance posture 131 files / 674 tests matches guard) > **Review trigger:** Subsystem maturity labels change. Content last changed: 2026-05-31 (TUI evolution Phase A-C + doc review); the 2026-09-16 review above confirmed labels unchanged since. @@ -80,7 +80,7 @@ Scale reference (internal engineering, not market validation): ## Honest External Posture - Public repo activity ≠ production validation. -- **Internal acceptance coverage is strong** (131 test files across `tests/acceptance/`, 673 collected tests; counts pinned by `tests/test_docs_consistency.py`). +- **Internal acceptance coverage is strong** (131 test files across `tests/acceptance/`, 674 collected tests; counts pinned by `tests/test_docs_consistency.py`). - External adoption signals (stars, forks, production references) remain early — do not infer enterprise readiness from architecture alone. ## Related Docs diff --git a/docs/retrospective/06-action-register.md b/docs/retrospective/06-action-register.md index 2d30d99f..52892cc5 100644 --- a/docs/retrospective/06-action-register.md +++ b/docs/retrospective/06-action-register.md @@ -42,6 +42,10 @@ Legend: ✅ Done (committed) · 🟡 In progress · ⬜ Not started · 🔵 Alre | U-P1-4 | UX | Reconcile stale dated documentation: add supersession notes to `tui-daily-driver-guide.md:54,67`, `tui-chat-reference.md:36-37`, and `run-evidence-and-audit-guide.md:82`, or update them. | As listed | No current-truth contradictions remain; docs-consistency CI passes. | ✅ | Current TUI cost, controller, undo, task-forwarding, root, and resume facts reconciled across operator, troubleshooting, and module docs on 2026-06-21; docs consistency and focused TUI tests pass | | U-P1-5 | UX | Remove `--no-tui` from all module documentation or implement it. | `docs/modules/cli/*`; `docs/modules/tui/inspection.md`; `docs/modules/chat_session_controller/api.md`; `docs/decisions/trade-offs.md:101` | Documentation matches code; either `rg --no-tui docs/` has no matches or the flag exists. | ✅ | This session — removed from 7 files; `rg --no-tui docs/` now returns only retrospective references (which document the removal) | | U-P1-6 | UX | Add a confirmation gate before chat mode forwards typos: `unknown command "x"; send as task? [y/N]`. | `teaagent/tui/_commands.py:396-397` | A typo no longer silently becomes an LLM task; add a test. | ✅ | Unknown commands require explicit confirmation before task forwarding; `tests/tui/test_tui_typo_confirmation.py` | +| G-P1-4 | Governance | Make a matched deny preset terminal in `prompt` mode: `POLICY_DENIED` must not fall through to interactive approval, matching the owner decision "block in all modes" (`docs/reviews/dogfood-g1-g22-2026-09-15-risk.md:22`). | `teaagent/runner/_governed_execution.py:314-335`; `teaagent/approval/manager.py:987` | An approving handler is never offered a deny-matched call; the tool does not run; `tool_call_blocked` records `policy_denied`. | ✅ | DENY-OVERRIDE, `docs/backlog-priority.md` | +| G-P1-5 | Governance | Wire a per-batch cancel token into subagent batches so a timed-out batch stops its running children at their next step instead of letting them keep writing after the batch reported a timeout. | `teaagent/subagents/_tools.py:315-370`; `teaagent/subagents/_manager.py:165-185`; `teaagent/runner/_core.py:907` | After the batch deadline a running child makes no new tool call and the cancellation is audited. | ✅ | WS2-002 residual, `docs/backlog-priority.md` | +| U-P1-7 | UX | Extend the operator cockpit state with branch, undo availability, last run id and active rules (M4 cockpit carve-out, DR-006 `owner-override: co-maintainer dogfood`). | `teaagent/cockpit.py:367` | A headless test asserts the four fields are exposed and rendered. | ✅ | DU-011, `docs/backlog-priority.md` | +| U-P1-8 | UX | Make the long-run heartbeat record current tool, elapsed time, cost and next checkpoint (M4 background-lifecycle carve-out, DR-006 `owner-override: co-maintainer dogfood`). | `teaagent/heartbeat.py:40-52` | A test asserts the heartbeat audit payload carries the four fields. | ✅ | DU-014, `docs/backlog-priority.md` | ## P2 - Cleanup (Polish/Debt/Defense in Depth) @@ -95,7 +99,7 @@ Legend: ✅ Done (committed) · 🟡 In progress · ⬜ Not started · 🔵 Alre | --- | --- | --- | | P0 | 9 | Security 3, Architecture 2, UX 2, Governance 2 | | P2 | 36 | Security 11, Governance 10, Architecture 7, UX 8 | -| **Total** | **63** | | +| **Total** | **67** | | ### Status Summary | ✅ Done (verified) | 61 | S-P0-1, S-P0-2, S-P0-3, A-P0-1, A-P0-2, U-P0-1, U-P0-2, G-P0-1, G-P0-2; S-P1-1, S-P1-2, S-P1-3, S-P1-4, G-P1-1, G-P1-2, G-P1-3, A-P1-1, A-P1-2, A-P1-3, A-P1-4, A-P1-5, U-P1-1, U-P1-2, U-P1-3, U-P1-4, U-P1-5, U-P1-6; S-P2-1, S-P2-4, S-P2-5, S-P2-6, S-P2-7, S-P2-8, S-P2-9, S-P2-10, S-P2-11, G-P2-1, G-P2-2, G-P2-3, G-P2-5, G-P2-6, G-P2-7, G-P2-8, G-P2-9, G-P2-10, G-P2-11, G-P2-12, G-P2-13, A-P2-1, A-P2-2, A-P2-3, A-P2-5, A-P2-6, A-P2-7, U-P2-1, U-P2-2, U-P2-3, U-P2-4, U-P2-5, U-P2-6, U-P2-7, U-P2-8 | diff --git a/docs/reviews/98-risk.md b/docs/reviews/98-risk.md new file mode 100644 index 00000000..03ab9ed6 --- /dev/null +++ b/docs/reviews/98-risk.md @@ -0,0 +1,51 @@ +# Risk Report — PR 98: terminal deny presets, batch cancel token, cockpit and heartbeat fields + +> **Claim class:** Risk note for one PR. Not current truth. +> **Last reviewed:** 2026-10-27 + +**Date:** 2026-10-27 +**Scope:** G-P1-4, G-P1-5, U-P1-7, U-P1-8 (`docs/retrospective/06-action-register.md`) +**High-risk path touched:** `teaagent/runner/_core.py` (per `scripts/high_risk_paths.yaml`). The approval change in `teaagent/runner/_governed_execution.py` is not on that list but changes approval behavior, so it is assessed here too. + +## What changes + +1. **Deny presets are terminal in `prompt` mode (G-P1-4).** `authorize_tool_call` caught every `ToolPermissionError` and, for a destructive tool in `prompt` mode, offered it to the interactive approval handler. A matched deny preset (`POLICY_DENIED`, `teaagent/approval/manager.py:987`) therefore ran when a person approved. It now goes to `record_blocked` and re-raises before the handler. Every other reason code keeps its current path. +2. **Subagent batches cancel their children at the deadline (G-P1-5).** Each batch creates a `threading.Event`, passes it to its children as `cancel_token`, and sets it at the deadline and on every exit. A set parent token is copied to the batch token while the batch waits. `AgentRunner` already checked the token at the top of each iteration; it now also checks it right before dispatching a tool decision (3 lines in `_core.py`). +3. **Cockpit run context (U-P1-7).** `ControlCockpitState` gains branch, undo availability, last run id and active rules; the TUI renders them. +4. **Heartbeat progress (U-P1-8).** Heartbeat ticks can carry tool, in-flight flag, elapsed seconds, cost and last checkpoint from an allow-listed provider. + +## Risk assessment + +- **Blast radius.** + - (1) Only calls that match an explicit deny grant change outcome, and only in `prompt` mode with a destructive tool and an approval handler. Allow, workspace-write, read-only and handler-less prompt runs already blocked. + - (2) Only `register_subagent_tools` batches pass the new token; swarm, tournament and team callers still inherit the parent token unchanged. + - (3) and (4) Additive read-only observability. +- **Failure modes.** + - (1) A user who relied on approving a denied call at the prompt is now blocked. That behavior contradicted `docs/cli.md:923`, `docs/permission-and-approval-playbook.md:63-64` and the owner decision in `docs/reviews/dogfood-g1-g22-2026-09-15-risk.md:22`. The escape hatch is `teaagent approval revoke `, pinned by a falsifier test. + - (2) A child stops at its next check, so a tool call already dispatched may still finish; the guarantee is "no new tool call after the deadline", not "no side effect after the deadline". A cancelled child records the existing `run_failed` (`system`, "run cancelled by cancel token"). The batch wait loop now wakes every 50 ms to poll the parent token. + - (3) Each source is wrapped: a failure logs, records an error string and shows `unknown`, never a silent pass (A-P0-2). + - (4) A raising provider sets `progress_unavailable` and the tick still records; the provider cannot overwrite `tick` or `interval_seconds`. +- **Thin harness.** `runner/_core.py` grows from 1,062 to 1,065 lines; god-module exemptions stay at 18. No new framework, event type or dependency: the cancel path reuses `cancel_token` and `RunCancelledError`. +- **Not changed.** `JIT_NO_APPROVAL`, `MULTISIG_NO_QUORUM` and the protected-path codes (`WORKSPACE_WRITE_MODE`, `SYMLINK_BLOCKED`, `SKILL_WRITE_BLOCKED`) still reach the approval prompt for destructive tools in `prompt` mode. No owner decision on record covers them; flagged in the PR for the owner. + +## Dry run and rollback + +- **Dry run.** Full suite on the PR head with the locked toolchain: 6830 passed, 34 skipped, 0 failed. All `lint` and `governance-gate` CI steps were run locally and exit 0. Each behavior change has a test that failed on the base: the approving handler ran the denied tool, and 4 tool calls landed after the batch deadline. +- **Rollback.** Each change is one commit and reverts independently: `801223dd` (deny), `98fa5e59` (cancel token), `57d84d70` (cockpit), `232df373` (heartbeat). Reverting `801223dd` restores prompt-mode override; reverting `98fa5e59` restores the previous behavior where abandoned children keep running. No data migration and no stored-state format change. + +## Tests + +- `tests/test_deny_preset_runtime_enforcement.py`: + - the approving handler is never called and the tool never runs; + - `tool_call_blocked` records `policy_denied`; + - `approval why-denied` describes it; + - revoking the grant lets the same handler run the tool. +- `tests/test_subagent_batch_cancel_token.py`: + - at most one already-dispatched tool call after the batch returns; + - a parent cancel mid-batch stops the children. +- `tests/test_control_cockpit.py` and `tests/acceptance/test_cockpit_snapshot_flow.py`: the four fields, plus the no-git, no-run and failure cases. +- `tests/test_heartbeat_progress_fields.py`: payload fields, backward compatibility without a provider, and one real run. + +## Decision + +Proceed. Change (1) aligns code with an owner decision already on record rather than making a new one. Change (2) makes the batch's "timed out" result true by reusing an existing primitive. Changes (3) and (4) are inside the DR-006 M4 cockpit and background-lifecycle carve-out. Human review: the PR is not merged without the owner's review; the protected-path question stays with the owner. diff --git a/docs/roadmap-status.md b/docs/roadmap-status.md index ae737367..407f1091 100644 --- a/docs/roadmap-status.md +++ b/docs/roadmap-status.md @@ -10,6 +10,8 @@ > > **Last reviewed:** 2026-10-23 (architecture rethink recorded at [reviews/architecture-rethink-2026-10-23.md](reviews/architecture-rethink-2026-10-23.md); no horizon, goal, milestone, or gate status changes. The 2026-10-09 review remains the last status-changing pass: ADR-0031 extended to 2026-11-15 bound to a B1 session, ADR-0043 packet prepared, Phase 0 repairs landed) +**Update 2026-10-27 (decision packets → fixes):** four read-only decision packets re-checked the remaining owner items against code. Two were already decided: the owner chose deny presets "block in all modes" (dogfood G1 risk record), so the prompt-mode override was a defect, now fixed (G-P1-4: `POLICY_DENIED` is terminal; `approval revoke` is the escape hatch); and the DR-006 M4 carve-out already covers cockpit and background-lifecycle work, so DU-011 (cockpit shows branch, undo, last run, active rules; U-P1-7) and DU-014 (heartbeat records tool, elapsed, cost, checkpoint; U-P1-8) were built. The WS2-002 residual was fixed with the runner's existing `cancel_token` (G-P1-5). PHASE-01…06 and six DU items carry PROPOSED verdicts in `backlog-priority.md`. No horizon status moved. + **Update 2026-10-26 (governance-gap follow-ups):** fixed WS2-002 (subagent batch timeout now bounds wall-clock) and DU-010A (`agent tick --max-cost-cents 0` no longer becomes a zero-spend cap); pinned DU-016 (unknown MCP tools always reach `ApprovalPolicy`, no bypass) and dogfood G1 (deny presets block at runtime, `tests/test_deny_preset_runtime_enforcement.py`, mutation-checked) with tests; the telemetry test no longer reads OpenTelemetry internals; a bare `approval deny ` now exits with a classified usage error naming `--path-glob`/`--command-prefix` instead of "Unexpected error"; `approval check`/`explain` no longer leak the argparse `command` value into the evaluated arguments; the operations manual now refuses queued calls with `approval reject `. Shipped in TeaEntityLab/teaAgent#97. New owner question DENY-OVERRIDE: in prompt mode an approving person can override a deny preset (see `backlog-priority.md`). No priority or horizon status moved. **Update 2026-10-26 (roadmap reconciliation):** doc-only pass, no priority or status of any open item moved. (1) The H1 caveat no longer calls dogfood findings G23–G38 "unadjudicated": the owner adjudicated them "Yes for all" on 2026-09-16 and the fixes landed, with G34 held under ADR-0043. (2) The 2026-09-15 deny-preset (G1) and undo/rollback (G2) entries are annotated with resolution evidence; G1 has no dedicated automated test. (3) A note above the North-Star Goals table separates goal IDs G1–G6 from dogfood finding IDs G1–G38. (4) Governance-plan status for ADOPTION-ROADMAP A1–A3 (A4 deferred) and SURF-010 steps 1–5 is recorded beside the GOV rows. Release evidence is to be regenerated separately. @@ -36,7 +38,7 @@ **Update 2026-09-15 (approval-shadow observation bias + allow-mode run):** drove an `allow`-mode run — the destructive call completed the full lifecycle (`tool_call_requested`/`started`/`completed`, file written) but emitted **no** `h4_governance_shadow`. Fourth residual gap recorded for owner adjudication: **the approval shadow only observes prompt-mode runs** — `evaluate_approval_policy_shadow` is invoked inside `handle_approval_request`, which only runs when `can_request_approval` is true (PROMPT mode + destructive). In `allow`/`workspace-write`/`read-only` modes the shadow never fires, so H4 evidence is systematically biased toward prompt-mode runs and blind to the modes where enforcement matters most. Whether the shadow should observe every destructive call regardless of mode is a semantic boundary (observation scope + audit volume), so recorded not fixed. Packet regenerated: 5 observed / 13 reachable runs / 1 denial candidate. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Update 2026-09-15 (observability + lifecycle surfaces verified):** exercised the H4 control-plane/audit surfaces on the dogfood runs — all healthy. `agent resume` completed the pending→resume→auto-approve→complete lifecycle (`auto_approved_call_id` recorded; consistent with the observation-bias finding, no `tool_call_approved`/shadow on the auto-approve path). `audit verify` reports **VALID hash chain** on all 7 dogfood runs (zero gaps/modifications/insertions). `runs export` (completeness `ok:true`, full trace), `runs replay` (dry-run, `tools_used`), `cockpit` (`pending_count:6`, `quarantine_count:28`, context health green), `daily` (flags pending approvals), and `runs list` all work. `origin: dogfood` correctly recorded in `runs-index.jsonl` (B-08) — owner can cleanly filter dogfood vs organic runs. `memory_write_quarantined` fired correctly (provenance gate quarantined the subagent's auto-curated memory write). No new gaps; these surfaces are healthy. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Update 2026-09-15 (approval lifecycle completed + provenance fix):** resolved all 6 pending approvals via `teaagent approval approve` — the canonical `tool_call_pending_approval`→`tool_call_approved` lifecycle now exercised organically on 8 runs. Found and fixed a provenance gap (`2f144102`): the CLI approve path recorded `tool_call_approved` with only `call_id`/`tool_name` — no `authority_type`/`approved_by` — so CLI-approved calls were indistinguishable from JIT-prompt approvals in the audit trail. Now records `authority_type='cli_approval'`, `approved_by='operator'` (consistent with `jit_prompt`/`auto_mode`/`preapproved_payload_digest`); verified on a dogfood run. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. -**Update 2026-09-15 (deny-preset enforcement gap — most serious finding):** exercised the deny path and found a documented safety control that is **advisory-only at runtime**. `teaagent approval deny ` registers a deny grant and `approval check`/`explain` correctly report `decision: deny` ("Matching deny grants block the tool call"), but the runtime path never blocks: `assert_allowed` → `check_preset` → `is_allowed`, which collapses `deny` and `prompt` both to `False`, so a matched deny grant is indistinguishable from "no preset" and falls through to `pending_approval` (prompt mode) or executes (allow mode). Verified: a scoped deny preset on `workspace_run_shell_mutate` did NOT block the matching call in either mode. Root cause: `check_preset` returns a bare bool that can't distinguish deny from no-match; `_resolve_decision` already exposes `decision` distinctly but the runtime path doesn't use it. **Not fixed** — enforcement change on `teaagent/approval/manager.py` (high-risk path) that alters when calls block; needs a risk report + owner sign-off. Test preset revoked. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Resolved:** the G1 fix is in code — `check_preset` now returns `deny` as a distinct value (`teaagent/approval/manager.py:757`) and `assert_allowed` raises `ToolPermissionError` on a matched deny (`teaagent/approval/manager.py:987-991`); live re-check recorded in `docs/work-log/dogfood-findings-2026-09-15.md` (G1 live). Regression test added 2026-10-26 (`tests/test_deny_preset_runtime_enforcement.py`, mutation-checked against the raise at `teaagent/approval/manager.py:987`). Caveat: in prompt mode an approving person can still override the deny (DENY-OVERRIDE, owner decision pending in `backlog-priority.md`). +**Update 2026-09-15 (deny-preset enforcement gap — most serious finding):** exercised the deny path and found a documented safety control that is **advisory-only at runtime**. `teaagent approval deny ` registers a deny grant and `approval check`/`explain` correctly report `decision: deny` ("Matching deny grants block the tool call"), but the runtime path never blocks: `assert_allowed` → `check_preset` → `is_allowed`, which collapses `deny` and `prompt` both to `False`, so a matched deny grant is indistinguishable from "no preset" and falls through to `pending_approval` (prompt mode) or executes (allow mode). Verified: a scoped deny preset on `workspace_run_shell_mutate` did NOT block the matching call in either mode. Root cause: `check_preset` returns a bare bool that can't distinguish deny from no-match; `_resolve_decision` already exposes `decision` distinctly but the runtime path doesn't use it. **Not fixed** — enforcement change on `teaagent/approval/manager.py` (high-risk path) that alters when calls block; needs a risk report + owner sign-off. Test preset revoked. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Resolved:** the G1 fix is in code — `check_preset` now returns `deny` as a distinct value (`teaagent/approval/manager.py:757`) and `assert_allowed` raises `ToolPermissionError` on a matched deny (`teaagent/approval/manager.py:987-991`); live re-check recorded in `docs/work-log/dogfood-findings-2026-09-15.md` (G1 live). Regression test added 2026-10-26 (`tests/test_deny_preset_runtime_enforcement.py`, mutation-checked against the raise at `teaagent/approval/manager.py:987`). Prompt-mode override closed 2026-10-27 (G-P1-4): a matched deny is never offered for approval. **Update 2026-09-15 (budget + plan-gate + sandbox boundaries verified):** exercised the remaining governance boundaries — all enforced correctly. **Budget**: `max_tool_calls=1` → `run_failed: tool-call budget exceeded` after exactly 1 call; `max_iterations=1` → `run_failed: iteration budget exceeded`; both with actionable messages. **Plan gate**: `run --from-plan --require-plan` blocked `workspace_write_file` with `tool_call_blocked`/`authority_type: policy_blocked` — "Intent drift: target outside the approved plan scope" (the write target wasn't in the plan's declared files). Intent-drift enforcement works. **Git sandbox**: `git_sandbox_started` fires per run; headless runs keep the `teaagent-sandbox-` branch for review and restore the tree (deliberate, not a bug) — but branches accumulate with no auto-prune (27 now; manual via doctor/experiment only), a minor housekeeping note. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Update 2026-09-15 (undo/rollback data-loss hazard — second serious finding):** exercised `teaagent undo` and found a destructive edge. `undo --preview` diffs `sandbox_branch → original_branch`, but `original_branch` (main) has moved forward since the run, so the preview shows **all later commits**, not just the run's writes — misleading. Worse, `GitBranchSandbox.rollback()` runs `git reset --hard HEAD` + `git clean -fd` **before** checkout, assuming HEAD is on the sandbox branch; but after a headless run `keep()` already restored `main`, so `undo` on a completed run would `reset --hard` + `clean -fd` **on `main` itself** — wiping uncommitted work — then `branch -D` the kept review branch. Verified by code read (`teaagent/sandbox/_git_branch.py:254-280`) + preview behavior; not executed (would destroy the working tree). **Not fixed** — `teaagent/sandbox/` is a high-risk path and the fix (guard rollback to only run on the sandbox branch, or scope the reset) changes destructive behavior; needs a risk report + owner sign-off. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. **Resolved:** `GitBranchSandbox.rollback()` now refuses when HEAD is not on the sandbox branch, before any `git reset --hard` (`teaagent/sandbox/_git_branch.py:255`, guard at `:280-290`); regression `tests/test_g2_g3_g22_focused.py::test_rollback_refuses_when_head_not_on_sandbox_branch` (code and test read 2026-10-26, test not re-run). The preview-base concern (diff against a moved `original_branch`) is only partly covered by `test_sandbox_stores_original_sha_for_preview_diff`. **Update 2026-09-15 (attach-resume provider gap):** exercised `agent attach --resume` on a pending run — it failed `run_failed: opencodezen-go requires OPENCODEZEN_API_KEY`. `attach` has no `--provider` flag and resumes with the configured default provider, not the original run's provider, so `attach --resume` **cannot resume any non-default-provider run** (it errors on credentials). `agent resume ` works (takes provider positionally). Recorded as a gap — the attach-resume path should inherit the run's recorded provider. `promotion_ready` stays `false`. No horizon/milestone status moved by this record. @@ -202,7 +204,7 @@ ratified wording. Adoption record: | G2 | Any run explained from one artifact | Pending — surface measured in CI (owner field validation still open) | `teaagent agent show ` plus run evidence summary (`tests/acceptance/test_run_evidence_summary_flow.py`); one-screen acceptance asserted by `test_run_receipt_answers_three_questions_within_one_screen` (B-02, 2026-09-12): why-allowed/what-changed/how-undo within `DEFAULT_PAGINATION_LINES` (50), passing in CI (verified 2026-09-16, 12/12 in file). Owner field validation (real-run readability) still open | | G3 | One event spine | Partial — rescoped by owner decision | M3 plan gate moved to a typed `RunEvent` spine interceptor cleanly; M4 approval/budget and M5 HookRegistry enforcement were assessed unsuitable for the spine/interceptor model and stay inline by owner-reviewed evidence (`docs/work-log/m4-budget-stays-inline-2026-06-13.md`, `docs/work-log/m5-hooks-observability-only-2026-06-13.md`). The spine's realized value is the observability read side, not wholesale relocation of mutating enforcement | | G4 | Extensible by hooks, not forks | Partial — observability wired; enforcement bridge held | `HookRegistry` exists and emits typed audit events (`TOOL_HOOK_PRE_MUTATION`, `TOOL_HOOK_POST_MUTATION`, etc.) surfaced by the M2 reader; `tests/acceptance/test_hook_lifecycle_flow.py` passes. Pre/post tool hooks mutate in-flight args/results in `teaagent/tools.py`; moving enforcement to spine consumers is unsuitable (same runtime-coupling finding as M4). Six session-lifecycle hooks are defined but not wired to production callers — see `m5-hooks-observability-only-2026-06-13.md` | -| G5 | Docs corpus carries its weight | Partial — corpus cost signal live + constitution cap enforced | `docs/generated/docs-aging-dashboard.md` `## Corpus Cost (G5 Signal)` reports total/live/archive counts and per-surface costs; `scripts/report_docs_aging.py` runs in CI. Current corpus (2026-10-26, after archiving 13 superseded/completed plans): 675 Markdown files, 674 in `docs-inventory.md` (8 constitution, 369 working, 297 archive). Constitution-tier ≤ 12 enforced by `check_docs_inventory` (`_CONSTITUTION_MAX_DOCS = 12`, pinned by `test_constitution_tier_cap_enforced_at_twelve`); archive-tier discipline is manual, not automated | +| G5 | Docs corpus carries its weight | Partial — corpus cost signal live + constitution cap enforced | `docs/generated/docs-aging-dashboard.md` `## Corpus Cost (G5 Signal)` reports total/live/archive counts and per-surface costs; `scripts/report_docs_aging.py` runs in CI. Current corpus (2026-10-27): 676 Markdown files, 675 in `docs-inventory.md` (8 constitution, 370 working, 297 archive). Constitution-tier ≤ 12 enforced by `check_docs_inventory` (`_CONSTITUTION_MAX_DOCS = 12`, pinned by `test_constitution_tier_cap_enforced_at_twelve`); archive-tier discipline is manual, not automated | | G6 | Tests prove behavior, not construction | **Complete — typing complete, ratchet absolute, zero flags** | 608 test files (as of 2026-09-15); all 458 backlog files classified 2026-09-13 (455 behavior / 82 contract / 39 adversarial / 31 lifecycle); `UNTYPED_BASELINE = 0` — `--fail-on untyped` in CI now fails on **any** new untyped file (VND-001 test file typed 2026-09-15, re-verified 0 untyped). All 71 `construction_only` + all 36 `no_assertions`/`assert_true`/`undocumented_skip` files strengthened 2026-09-13 (every assert verified against probed runtime behavior). Audit fully clean: `high_risk_files: 0`, `medium_risk_files: 0`, `construction_only_files: 0`, `placeholder_files: 0`. Raw suite counts are not a quality claim per harness-first §4.2 | ## Roadmap-Neutral Governance-Gap Intake - Effect Authority diff --git a/knowledge/teaagent-current/concepts/acceptance.md b/knowledge/teaagent-current/concepts/acceptance.md index 04994db6..13b4fe17 100644 --- a/knowledge/teaagent-current/concepts/acceptance.md +++ b/knowledge/teaagent-current/concepts/acceptance.md @@ -6,7 +6,7 @@ resource: "urn:teaagent:doc:docs/acceptance.md" tags: ["teaagent", "acceptance", "testing", "constitution"] teaagent: source_path: "docs/acceptance.md" - source_sha256: "eb345d401d1c0d655273f0e42b387e7460e6e0ce22e2f73bfdeae1e97a2317ef" + source_sha256: "175a778989b19522da5a2a677e0fca2cb217543d104bd8bcae35aa99dea297fd" docs_tier: "constitution" authority: "canonical" lifecycle: "current" diff --git a/knowledge/teaagent-current/concepts/roadmap-status.md b/knowledge/teaagent-current/concepts/roadmap-status.md index efb111bb..5043a112 100644 --- a/knowledge/teaagent-current/concepts/roadmap-status.md +++ b/knowledge/teaagent-current/concepts/roadmap-status.md @@ -6,7 +6,7 @@ resource: "urn:teaagent:doc:docs/roadmap-status.md" tags: ["teaagent", "roadmap", "status", "constitution"] teaagent: source_path: "docs/roadmap-status.md" - source_sha256: "19c400a3ad1ef1d29be9d356a7fa21e807909b45a8799827b89a241e8c1546f8" + source_sha256: "9df14453d98f2c08d1e137522a4d344ad332604a37254d0aebb57f8ae19f89b9" docs_tier: "constitution" authority: "canonical" lifecycle: "current" diff --git a/teaagent/chat_agent.py b/teaagent/chat_agent.py index 4e3fb59a..f6fa8647 100644 --- a/teaagent/chat_agent.py +++ b/teaagent/chat_agent.py @@ -2,6 +2,7 @@ import contextlib import threading +import time import warnings from collections.abc import Callable from dataclasses import dataclass @@ -9,7 +10,7 @@ from typing import Any, Optional, cast, overload from uuid import uuid4 -from teaagent.audit import AuditLogger +from teaagent.audit import AuditEvent, AuditLogger from teaagent.auto_mode import AutoModeConfig from teaagent.browser_tools import register_browser_tools from teaagent.budget import RunBudget @@ -22,7 +23,7 @@ ) from teaagent.context import ContextCompactor from teaagent.errors import ToolValidationError -from teaagent.heartbeat import Heartbeat +from teaagent.heartbeat import Heartbeat, ProgressProvider from teaagent.hooks import HookRegistry from teaagent.integration.run_contract import build_run_budget from teaagent.llm import ( @@ -697,10 +698,57 @@ def _create_runner_and_engine( return runner, engine +def _heartbeat_progress( + config: ChatAgentConfig, + audit_logger: AuditLogger, + engine: ModelDecisionEngine, +) -> ProgressProvider: + """Build the progress snapshot read by each heartbeat tick. + + The tool name and in-flight flag come from the runner's ``tool_call_*`` + audit events. The checkpoint is the last completed tool call, because the + runner saves its checkpoint right after each completion; it is None when + no checkpoint store is configured. Elapsed time counts from this call. + """ + started_at = time.monotonic() + + def _progress() -> dict[str, object]: + tool: Optional[str] = None + tool_in_flight = False + last_done: Optional[AuditEvent] = None + finished: set[object] = set() + for event in reversed(list(audit_logger.events)): + if event.event_type in ('tool_call_completed', 'tool_call_failed'): + finished.add(event.payload.get('call_id')) + if last_done is None and event.event_type == 'tool_call_completed': + last_done = event + elif event.event_type == 'tool_call_started' and tool is None: + tool = event.payload.get('tool_name') + tool_in_flight = event.payload.get('call_id') not in finished + if tool is not None and last_done is not None: + break + checkpoint = None + if last_done is not None and config.checkpoint_store is not None: + checkpoint = { + 'call_id': last_done.payload.get('call_id'), + 'at': last_done.created_at, + } + return { + 'tool': tool, + 'tool_in_flight': tool_in_flight, + 'elapsed_seconds': round(time.monotonic() - started_at, 3), + 'cost_cents': float(engine.usage.cost_cents), + 'checkpoint': checkpoint, + } + + return _progress + + def _setup_heartbeat( config: ChatAgentConfig, audit_logger: AuditLogger, run_id: str, + engine: ModelDecisionEngine, ) -> Optional[Heartbeat]: """Setup heartbeat monitoring if configured. @@ -714,6 +762,7 @@ def _setup_heartbeat( run_id, interval_seconds=config.heartbeat_seconds, liveness_root=config.root, + progress=_heartbeat_progress(config, audit_logger, engine), ) heartbeat.start() return heartbeat @@ -870,7 +919,7 @@ def _run_chat_agent_impl( run_id, audit_logger, ) - heartbeat = _setup_heartbeat(config, audit_logger, run_id) + heartbeat = _setup_heartbeat(config, audit_logger, run_id, engine) run_started_extra = _build_run_started_extra(runner, context_extra, config) return _execute_chat_run( task=task, diff --git a/teaagent/cli/_handlers/_ergonomics/approval.py b/teaagent/cli/_handlers/_ergonomics/approval.py index 0977f96e..d5a5fef6 100644 --- a/teaagent/cli/_handlers/_ergonomics/approval.py +++ b/teaagent/cli/_handlers/_ergonomics/approval.py @@ -158,6 +158,7 @@ def _build_explanation_summary(check_result: dict[str, Any]) -> str: 'multisig_no_quorum': 'Blocked by multi-signature quorum — not enough peer approvals.', 'auto_mode_blocked': 'Blocked by auto-mode tool restrictions.', 'missing_state': 'Blocked — no matching approval state found.', + 'policy_denied': 'Blocked by a deny grant (approval preset) — always terminal; not offered for approval. Remove it with `approval revoke `.', } diff --git a/teaagent/cockpit.py b/teaagent/cockpit.py index 1cc22f99..1dc98967 100644 --- a/teaagent/cockpit.py +++ b/teaagent/cockpit.py @@ -404,6 +404,24 @@ class ControlCockpitState: errors: list[str] = field(default_factory=list) last_updated: Optional[float] = None + # DU-011: run context. String sentinels: 'none' = source present but empty + # (or not a git repo), 'unknown' = source failed to load. + branch: str = 'unknown' # branch name, 'detached HEAD', or 'none' + undo: str = 'unknown' # available | none | unknown + last_run_id: str = 'unknown' # run id, 'none', or 'unknown' + # Non-expired approval grants as 'tool (scope)'; None means unknown. + active_rules: Optional[list[str]] = None + + +def _current_branch(root_path: Path) -> str: + """Return the checked-out branch name, or 'none' outside a git repository.""" + named = _run_git(['symbolic-ref', '--short', '-q', 'HEAD'], cwd=root_path) + if named is not None and named.stdout.strip(): + return named.stdout.strip() + if _run_git(['rev-parse', '--is-inside-work-tree'], cwd=root_path) is not None: + return 'detached HEAD' + return 'none' + def build_control_cockpit( root: Path | str, @@ -544,6 +562,47 @@ def build_control_cockpit( 'state': cost_state, } + # ── run context: branch, last run, undo, active rules (DU-011) ── + try: + cockpit.branch = _current_branch(root_path) + except Exception: + logger.exception('git branch lookup failed') + cockpit.errors.append('git branch lookup failed') + + try: + from teaagent.run_store import RunStore + from teaagent.run_undo import UndoJournal + + runs = RunStore(root_path) + recent_runs = runs.list_runs(limit=1) + cockpit.last_run_id = recent_runs[0].run_id if recent_runs else 'none' + undo_run = runs.latest_run_with_undo() + if undo_run is None: + cockpit.undo = 'none' + else: + journal = UndoJournal(root_path, path=runs.undo_path(undo_run)) + cockpit.undo = 'available' if journal.has_entries else 'none' + except Exception: + logger.exception('run/undo state load failed') + cockpit.last_run_id = 'unknown' + cockpit.undo = 'unknown' + cockpit.errors.append('run/undo state load failed') + + try: + from teaagent.ergonomics._approval_grants import _grant_expired + from teaagent.ergonomics._approval_state import ApprovalPresetStore + + grants = ApprovalPresetStore(root_path, readonly=True).list_grants() + cockpit.active_rules = [ + f'{grant.tool_name} ({grant.scope})' + for grant in grants + if not _grant_expired(grant) + ] + except Exception: + logger.exception('approval rules load failed') + cockpit.active_rules = None + cockpit.errors.append('approval rules load failed') + # ── context health (CTX-001) ── try: from teaagent.context_health import compute_context_health diff --git a/teaagent/heartbeat.py b/teaagent/heartbeat.py index 7f8aba8e..e47dbd92 100644 --- a/teaagent/heartbeat.py +++ b/teaagent/heartbeat.py @@ -2,14 +2,31 @@ import threading import time -from collections.abc import Callable +from collections.abc import Callable, Mapping from pathlib import Path from typing import Optional from teaagent.audit import AuditLogger +# Keys a progress provider may contribute to a heartbeat payload. Anything else +# (including ``tick`` and ``interval_seconds``) is ignored so a provider cannot +# corrupt the heartbeat's own counters. +PROGRESS_FIELDS = frozenset( + {'tool', 'tool_in_flight', 'elapsed_seconds', 'cost_cents', 'checkpoint'} +) + +# Returns the run's current progress snapshot, or None when nothing is known yet. +ProgressProvider = Callable[[], Optional[Mapping[str, object]]] + class Heartbeat: + """Periodic liveness ticker that records progress for a long-running run. + + When a ``progress`` provider is given, each tick also carries the fields + in ``PROGRESS_FIELDS`` that the provider returns. Without one, the payload + holds only ``tick`` and ``interval_seconds``. + """ + def __init__( self, audit: AuditLogger, @@ -18,6 +35,7 @@ def __init__( interval_seconds: float, sleep: Callable[[float], None] = time.sleep, liveness_root: Path | None = None, + progress: ProgressProvider | None = None, ) -> None: if interval_seconds <= 0: raise ValueError('interval_seconds must be positive') @@ -26,6 +44,7 @@ def __init__( self.interval_seconds = interval_seconds self._sleep = sleep self._liveness_root = liveness_root + self._progress = progress self._stop_event = threading.Event() self._thread: Optional[threading.Thread] = None self.tick_count = 0 @@ -42,8 +61,7 @@ def tick(self) -> None: self.audit.record( 'heartbeat', self.run_id, - tick=self.tick_count, - interval_seconds=self.interval_seconds, + **self._payload(), ) if self._liveness_root is not None: from teaagent.ergonomics.run_liveness import touch_liveness @@ -55,6 +73,23 @@ def tick(self) -> None: interval_seconds=self.interval_seconds, ) + def _payload(self) -> dict[str, object]: + payload: dict[str, object] = { + 'tick': self.tick_count, + 'interval_seconds': self.interval_seconds, + } + if self._progress is None: + return payload + try: + snapshot = self._progress() + except Exception: + # A broken progress source must never stop the liveness signal. + payload['progress_unavailable'] = True + return payload + if snapshot: + payload.update({k: v for k, v in snapshot.items() if k in PROGRESS_FIELDS}) + return payload + def start(self) -> None: if self._thread is not None: return diff --git a/teaagent/runner/_core.py b/teaagent/runner/_core.py index 8cd13fd3..4e82327c 100644 --- a/teaagent/runner/_core.py +++ b/teaagent/runner/_core.py @@ -971,6 +971,9 @@ def _execute_run_loop( raw_decision_preview=preview, ) + # decide() may have blocked for seconds; re-check before dispatch. + if self.cancel_token is not None and self.cancel_token.is_set(): + raise RunCancelledError('run cancelled by cancel token') tool_calls, context = self._execute_tool_decision( decision, context, diff --git a/teaagent/runner/_governed_execution.py b/teaagent/runner/_governed_execution.py index 0f6b6f97..379ac90c 100644 --- a/teaagent/runner/_governed_execution.py +++ b/teaagent/runner/_governed_execution.py @@ -27,6 +27,7 @@ from teaagent.budget_monitor import BudgetAction, BudgetMonitor from teaagent.errors import ( BudgetExceededError, + DenialReasonCode, RunCancelledError, ToolPermissionError, ) @@ -35,7 +36,6 @@ from ._events import RunEventType if TYPE_CHECKING: - from teaagent.errors import DenialReasonCode from teaagent.run_context import RunContext from ._core import AgentRunner @@ -322,6 +322,16 @@ def authorize_tool_call( annotations=annotations, run_id=run_id, ) + if exc_reason_code is DenialReasonCode.POLICY_DENIED: + # An explicit deny preset is terminal in every mode (owner decision + # "block in all modes"; docs/cli.md:923). Never offer it for approval. + runner.approval_manager.record_blocked( + approval_request=approval_request, + audit=runner.audit, + run_id=run_id, + reason_code=reason_code_str, + ) + raise if runner.approval_manager.can_request_approval(tool.annotations.destructive): approved = runner.approval_manager.handle_approval_request( approval_request=approval_request, diff --git a/teaagent/subagents/_manager.py b/teaagent/subagents/_manager.py index 69330e4a..f16fd28d 100644 --- a/teaagent/subagents/_manager.py +++ b/teaagent/subagents/_manager.py @@ -1,6 +1,7 @@ from __future__ import annotations import logging +import threading from dataclasses import replace from datetime import datetime, timezone from pathlib import Path @@ -172,6 +173,7 @@ def _build_subagent_config( def_used: str, approval_handler: Any, inherited_mode: Any, + cancel_token: Optional[threading.Event] = None, ) -> Any: return replace( parent_config, @@ -182,6 +184,11 @@ def _build_subagent_config( model=(sub_def.model if sub_def and sub_def.model else parent_config.model), permission_mode=inherited_mode, approval_handler=approval_handler, + # A batch passes its own token so the deadline can stop the child; other + # callers keep the parent's token. + cancel_token=( + cancel_token if cancel_token is not None else parent_config.cancel_token + ), ) @@ -244,6 +251,7 @@ def run_subagent( isolation: str = DEFAULT_SUBAGENT_ISOLATION, skill_path: Optional[str | Path] = None, skill_risk_level: Optional[Any] = None, + cancel_token: Optional[threading.Event] = None, ) -> dict[str, Any]: """Spawn and run an isolated subagent, returning a serializable result payload.""" if depth >= self._parent_config.max_subagent_depth: @@ -347,6 +355,7 @@ def run_subagent( def_used, approval_handler, inherited_mode, + cancel_token=cancel_token, ) return self._execute_subagent_run( diff --git a/teaagent/subagents/_tools.py b/teaagent/subagents/_tools.py index 38c43625..438c65c9 100644 --- a/teaagent/subagents/_tools.py +++ b/teaagent/subagents/_tools.py @@ -1,5 +1,6 @@ from __future__ import annotations +import threading import time from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait from typing import Any @@ -16,6 +17,9 @@ from teaagent.subagents._types import DEFAULT_SUBAGENT_ISOLATION from teaagent.tools import ToolAnnotations, ToolRegistry +# How often the batch loop re-checks the parent cancel token while waiting. +_BATCH_CANCEL_POLL_SECONDS = 0.05 + def register_subagent_tools( registry: ToolRegistry, @@ -311,10 +315,15 @@ def _run_one(task_obj: dict, batch_index: int) -> dict[str, Any]: max_tool_calls=_as_int(task_obj.get('max_tool_calls')), batch_index=batch_index, isolation=isolation, + cancel_token=batch_token, ) results: list[tuple[int, dict[str, Any]]] = [] deadline = time.monotonic() + max(timeout_seconds, 1) + # One token per batch. Children check it before every tool dispatch, so + # setting it at the deadline stops them instead of letting them run on. + batch_token = threading.Event() + parent_cancel = config.cancel_token # Not a context manager: its __exit__ calls shutdown(wait=True), which # would block until every running subagent finishes and defeat the deadline. pool = ThreadPoolExecutor(max_workers=max_workers) @@ -328,9 +337,11 @@ def _run_one(task_obj: dict, batch_index: int) -> dict[str, Any]: remaining = deadline - time.monotonic() if remaining <= 0: break + if parent_cancel is not None and parent_cancel.is_set(): + batch_token.set() done, pending = wait( pending, - timeout=remaining, + timeout=min(remaining, _BATCH_CANCEL_POLL_SECONDS), return_when=FIRST_COMPLETED, ) for future in done: @@ -354,16 +365,18 @@ def _run_one(task_obj: dict, batch_index: int) -> dict[str, Any]: results.append((idx, _subagent_error(str(exc)))) continue else: - # Python threads cannot be killed and subagent runs expose no - # cooperative cancel hook, so the still-running thread is - # abandoned: it keeps running in the background and its result - # is ignored. + # Python threads cannot be killed. The batch token is already + # set, so the child stops at its next cancel check; only a tool + # call it had already dispatched may still finish. Its result + # is discarded. message = ( f'batch timeout after {timeout_seconds}s while task was ' - 'running; result discarded' + 'running; cancel requested, result discarded' ) results.append((idx, _subagent_error(message))) finally: + # Set on every exit path so no child outlives the batch. + batch_token.set() # Do not wait for abandoned threads; completed results were already # collected above. pool.shutdown(wait=False, cancel_futures=True) diff --git a/teaagent/tui/core.py b/teaagent/tui/core.py index 7f4d3c7e..5172fd80 100644 --- a/teaagent/tui/core.py +++ b/teaagent/tui/core.py @@ -372,6 +372,16 @@ def _print_state_panel(self) -> None: f' Approval: {app.get("pending_count", 0)} pending, {app.get("blocked_count", 0)} blocked, mode={app.get("mode", "?")}' ) + # Run context (DU-011) + print(f' Branch: {cc.branch}') + print(f' Undo: {cc.undo}') + print(f' Last run: {cc.last_run_id}') + if cc.active_rules is None: + rules_label = 'unknown' + else: + rules_label = ', '.join(cc.active_rules) or 'none' + print(f' Rules: {rules_label}') + # Cost cost = cc.cost spent = cost.get('spent_cents', 0.0) diff --git a/tests/acceptance/test_cockpit_snapshot_flow.py b/tests/acceptance/test_cockpit_snapshot_flow.py index bbc6e865..eb2d4414 100644 --- a/tests/acceptance/test_cockpit_snapshot_flow.py +++ b/tests/acceptance/test_cockpit_snapshot_flow.py @@ -64,3 +64,13 @@ def test_cli_cockpit_emits_snapshot_contract() -> None: assert code == 0 payload = json.loads(out.getvalue()) assert payload['schema_version'] == COCKPIT_SNAPSHOT_SCHEMA_VERSION + + +def test_build_cockpit_snapshot_control_exposes_run_context() -> None: + """DU-011: the shared snapshot carries branch, undo, last run and rules.""" + with tempfile.TemporaryDirectory() as tmp: + control = build_cockpit_snapshot(tmp)['control'] + assert control['branch'] == 'none' + assert control['undo'] == 'none' + assert control['last_run_id'] == 'none' + assert control['active_rules'] == [] diff --git a/tests/test_control_cockpit.py b/tests/test_control_cockpit.py index a58b3f96..c1924ee2 100644 --- a/tests/test_control_cockpit.py +++ b/tests/test_control_cockpit.py @@ -1,6 +1,7 @@ from __future__ import annotations # test-type: behavior +import subprocess import tempfile import time from io import StringIO @@ -364,3 +365,151 @@ def test_refresh_control_cockpit_handles_errors() -> None: tui._refresh_control_cockpit() assert tui._control_cockpit is None + + +# ── DU-011: branch, undo, last run, and active rules ───────────────────────── + + +def _git(root: Path, *args: str) -> None: + subprocess.run( + [ + 'git', + '-c', + 'user.name=test', + '-c', + 'user.email=test@example.com', + '-c', + 'commit.gpgsign=false', + *args, + ], + cwd=root, + check=True, + capture_output=True, + text=True, + ) + + +def _init_repo_on_branch(root: Path, branch: str) -> None: + _git(root, 'init', '-q', '-b', branch) + (root / 'README.md').write_text('seed\n', encoding='utf-8') + _git(root, 'add', 'README.md') + _git(root, 'commit', '-q', '-m', 'seed') + + +def _seed_run_with_undo(root: Path, run_id: str) -> None: + """Run a real agent write so the run record and undo journal both exist.""" + from conftest import FakeAdapter + + from teaagent.chat_agent import ChatAgentConfig, run_chat_agent + from teaagent.run_store import RunStore + from teaagent.run_undo import UndoJournal + from teaagent.types import AuditLogger + + store = RunStore(root) + audit = AuditLogger(path=store.run_path(run_id)) + journal = UndoJournal(root) + audit.add_sink(journal) + adapter = FakeAdapter( + [ + ( + '{"type":"tool","tool_name":"workspace_write_file",' + '"arguments":{"path":"notes.txt","content":"new\\n"},"call_id":"w1"}' + ), + '{"type":"final","content":"done"}', + ] + ) + run_chat_agent( + ChatAgentConfig.from_root( + root, allow_destructive=True, max_iterations=4, max_tool_calls=4 + ), + 'write notes', + adapter=adapter, + audit=audit, + run_id=run_id, + ) + # Mirror production (cli/execution.py): persist the journal for the run. + journal.save_to(store.undo_path(run_id)) + + +def test_cockpit_exposes_branch_undo_last_run_and_rules(tmp_path: Path) -> None: + from teaagent.ergonomics._approval_state import ApprovalPresetStore + + _init_repo_on_branch(tmp_path, 'feature/cockpit-fields') + _seed_run_with_undo(tmp_path, 'run-cockpit-001') + ApprovalPresetStore(tmp_path).grant('workspace_read_file', scope='session') + + cockpit = build_control_cockpit(tmp_path) + + assert cockpit.branch == 'feature/cockpit-fields' + assert cockpit.undo == 'available' + assert cockpit.last_run_id == 'run-cockpit-001' + assert cockpit.active_rules == ['workspace_read_file (session)'] + assert cockpit.errors == [] + + +def test_cockpit_panel_renders_branch_undo_last_run_and_rules(tmp_path: Path) -> None: + from teaagent.ergonomics._approval_state import ApprovalPresetStore + from teaagent.tui import TeaAgentTUI + + _init_repo_on_branch(tmp_path, 'feature/cockpit-panel') + _seed_run_with_undo(tmp_path, 'run-cockpit-002') + ApprovalPresetStore(tmp_path).grant('workspace_read_file', scope='session') + + tui = TeaAgentTUI(input_fn=lambda _p: 'exit', output_fn=lambda _: None) + tui._control_cockpit = build_control_cockpit(tmp_path) + output = _render_cockpit(tui) + + assert 'Branch: feature/cockpit-panel' in output + assert 'Undo: available' in output + assert 'Last run: run-cockpit-002' in output + assert 'Rules: workspace_read_file (session)' in output + + +def test_cockpit_run_context_is_none_without_git_or_runs(tmp_path: Path) -> None: + cockpit = build_control_cockpit(tmp_path) + + assert cockpit.branch == 'none' + assert cockpit.undo == 'none' + assert cockpit.last_run_id == 'none' + assert cockpit.active_rules == [] + assert cockpit.errors == [] + + +def test_cockpit_run_context_renders_none_explicitly(tmp_path: Path) -> None: + from teaagent.tui import TeaAgentTUI + + tui = TeaAgentTUI(input_fn=lambda _p: 'exit', output_fn=lambda _: None) + tui._control_cockpit = build_control_cockpit(tmp_path) + output = _render_cockpit(tui) + + assert 'Branch: none' in output + assert 'Undo: none' in output + assert 'Last run: none' in output + assert 'Rules: none' in output + + +def test_cockpit_run_context_is_unknown_when_sources_fail(tmp_path: Path) -> None: + from teaagent.tui import TeaAgentTUI + + (tmp_path / '.teaagent').mkdir() + (tmp_path / '.teaagent' / 'approvals.json').write_text( + '{not json', encoding='utf-8' + ) + with patch( + 'teaagent.run_store.RunStore.list_runs', + side_effect=OSError('runs unreadable'), + ): + cockpit = build_control_cockpit(tmp_path) + + assert cockpit.last_run_id == 'unknown' + assert cockpit.undo == 'unknown' + assert cockpit.active_rules is None + assert 'run/undo state load failed' in cockpit.errors + assert 'approval rules load failed' in cockpit.errors + + tui = TeaAgentTUI(input_fn=lambda _p: 'exit', output_fn=lambda _: None) + tui._control_cockpit = cockpit + output = _render_cockpit(tui) + assert 'Last run: unknown' in output + assert 'Undo: unknown' in output + assert 'Rules: unknown' in output diff --git a/tests/test_deny_preset_runtime_enforcement.py b/tests/test_deny_preset_runtime_enforcement.py index 8d11bb24..8b3ffcd3 100644 --- a/tests/test_deny_preset_runtime_enforcement.py +++ b/tests/test_deny_preset_runtime_enforcement.py @@ -22,14 +22,21 @@ - Without a deny preset the same flow executes the tool (control), so the blocking assertions cannot pass vacuously. -Not pinned here: in ``prompt`` mode with an ``approval_handler`` that approves, the -runner's ToolPermissionError path (``teaagent/runner/_governed_execution.py:314-335``) -still asks the handler, and an approving handler runs a deny-matched call. Whether -an interactive override should beat a deny preset is an open owner decision. +- Deny preset in ``prompt`` mode with an approving ``approval_handler``: the call is + still blocked and the handler is never consulted. This is the owner decision + "block in all modes" (docs/reviews/dogfood-g1-g22-2026-09-15-risk.md:22; + docs/cli.md:923 "Matched deny grants hard-block in every mode"). The runner's + ToolPermissionError path (``teaagent/runner/_governed_execution.py``) records a + ``policy_denied`` block and re-raises for ``POLICY_DENIED`` instead of offering + interactive approval. Removing the deny grant (``approval revoke``) is the only + way to let such a call run again; the falsifier test pins that escape hatch. """ from __future__ import annotations +import io +import json +from contextlib import redirect_stdout from pathlib import Path from typing import Any @@ -38,7 +45,14 @@ from teaagent.cli import main from teaagent.ergonomics._approval_state import ApprovalPresetStore from teaagent.policy import ApprovalPolicy -from teaagent.runner import AgentRunner, ApprovalRequest, FinalAnswer, ToolRequest +from teaagent.run_store import RunStore +from teaagent.runner import ( + AgentRunner, + ApprovalRequest, + FinalAnswer, + RunResult, + ToolRequest, +) from teaagent.types import ( AuditLogger, PermissionMode, @@ -78,11 +92,18 @@ def _write(arguments: dict[str, Any]) -> dict[str, Any]: return registry -def _register_deny_via_cli(root: Path, capsys: pytest.CaptureFixture[str]) -> None: - """Register the deny grant exactly as the G1 repro does (CLI, not the store API).""" +def _register_deny_via_cli(root: Path, capsys: pytest.CaptureFixture[str]) -> str: + """Register the deny grant exactly as the G1 repro does (CLI, not the store API). + + Returns the new grant's ``grant_id`` so callers can revoke it. + """ rc = main(['approval', 'deny', _TOOL, '--root', str(root), '--path-glob', '*']) assert rc == 0 - assert '"scope": "deny"' in capsys.readouterr().out + out = capsys.readouterr().out + assert '"scope": "deny"' in out + grant_id = json.loads(out)['grant_id'] + assert isinstance(grant_id, str) + return grant_id def _run_tool_call( @@ -90,9 +111,10 @@ def _run_tool_call( *, permission_mode: PermissionMode, approval_handler: Any = None, + audit: AuditLogger | None = None, ) -> tuple[Any, list[dict[str, Any]], AuditLogger]: calls: list[dict[str, Any]] = [] - audit = AuditLogger() + audit = audit if audit is not None else AuditLogger() policy = ApprovalPolicy( permission_mode=permission_mode, approval_store=ApprovalPresetStore(root), @@ -213,3 +235,109 @@ def test_without_deny_grant_allow_mode_runs_tool(tmp_path: Path) -> None: assert calls == [_ARGS] assert 'tool_call_started' in _event_types(audit) assert 'tool_call_blocked' not in _event_types(audit) + + +def test_deny_grant_blocks_approving_handler_in_prompt_mode( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """(d) G-P1-4: a deny preset is terminal in ``prompt`` mode. + + The handler would approve, but a matched deny must never reach it. Before the + fix, ``_governed_execution.py`` offered the ToolPermissionError to the handler + for destructive tools, and the approving handler ran the denied tool. + """ + _register_deny_via_cli(tmp_path, capsys) + handler_calls: list[ApprovalRequest] = [] + + def approve(req: ApprovalRequest) -> bool: + handler_calls.append(req) + return True + + result, calls, audit = _run_tool_call( + tmp_path, permission_mode=PermissionMode.PROMPT, approval_handler=approve + ) + + assert handler_calls == [] + assert calls == [] + assert result.status != 'completed' + event_types = _event_types(audit) + assert 'tool_call_started' not in event_types + assert 'tool_call_approved' not in event_types + assert 'tool_call_blocked' in event_types + assert 'policy_denied' in _reason_codes(audit, ('tool_call_blocked',)) + + +def test_why_denied_renders_policy_denied_description( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """(e) ``approval why-denied`` renders ``policy_denied`` with a real description.""" + _register_deny_via_cli(tmp_path, capsys) + run_id = 'run-g1-4-why-denied' + store = RunStore(tmp_path) + audit = store.audit_logger(run_id) + + def approve(_req: ApprovalRequest) -> bool: + return True + + _run_tool_call( + tmp_path, + permission_mode=PermissionMode.PROMPT, + approval_handler=approve, + audit=audit, + ) + store.logger_for_result( + RunResult( + run_id=run_id, + final_answer=None, + iterations=1, + tool_calls=0, + status='failed', + ), + audit, + ) + + out = io.StringIO() + with redirect_stdout(out): + code = main(['approval', 'why-denied', run_id, '--root', str(tmp_path)]) + + assert code == 0 + rendered = out.getvalue() + assert 'Reason: policy_denied' in rendered + assert 'Unknown denial reason' not in rendered + assert 'approval preset' in rendered + + +def test_revoking_deny_grant_lets_approving_handler_run_tool( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """(f) Falsifier: ``approval revoke `` is the escape hatch. + + With the deny present the approving handler is never consulted. After revoke, + a new run with the same approving handler executes the tool, which shows the + block came from the deny grant and nothing else. + """ + grant_id = _register_deny_via_cli(tmp_path, capsys) + handler_calls: list[ApprovalRequest] = [] + + def approve(req: ApprovalRequest) -> bool: + handler_calls.append(req) + return True + + _, blocked_calls, _ = _run_tool_call( + tmp_path, permission_mode=PermissionMode.PROMPT, approval_handler=approve + ) + assert blocked_calls == [] + assert handler_calls == [] + + rc = main(['approval', 'revoke', grant_id, '--root', str(tmp_path)]) + assert rc == 0 + assert json.loads(capsys.readouterr().out)['status'] == 'revoked' + + result, calls, audit = _run_tool_call( + tmp_path, permission_mode=PermissionMode.PROMPT, approval_handler=approve + ) + assert result.status == 'completed' + assert calls == [_ARGS] + assert len(handler_calls) == 1 + assert 'tool_call_started' in _event_types(audit) + assert 'policy_denied' not in _reason_codes(audit, _BLOCKING_EVENTS) diff --git a/tests/test_heartbeat_progress_fields.py b/tests/test_heartbeat_progress_fields.py new file mode 100644 index 00000000..cf20a9d3 --- /dev/null +++ b/tests/test_heartbeat_progress_fields.py @@ -0,0 +1,130 @@ +# test-type: behavior +from __future__ import annotations + +import time +from typing import Any, Optional + +from conftest import FakeAdapter + +from teaagent import AuditLogger, ChatAgentConfig, Heartbeat, RunStore, run_chat_agent + + +def _heartbeat_payloads(audit: AuditLogger) -> list[dict[str, Any]]: + return [e.payload for e in audit.events if e.event_type == 'heartbeat'] + + +def test_tick_payload_carries_tool_elapsed_cost_and_checkpoint() -> None: + audit = AuditLogger() + checkpoint = {'call_id': 'read-1', 'at': '2026-10-27T00:00:00+00:00'} + + def provider() -> Optional[dict[str, Any]]: + return { + 'tool': 'workspace_read_file', + 'elapsed_seconds': 12.5, + 'cost_cents': 3.2, + 'checkpoint': checkpoint, + } + + beat = Heartbeat( + audit, + 'run-progress', + interval_seconds=30.0, + progress=provider, + ) + beat.tick() + + (payload,) = _heartbeat_payloads(audit) + assert payload['tick'] == 1 + assert payload['interval_seconds'] == 30.0 + assert payload['tool'] == 'workspace_read_file' + assert payload['elapsed_seconds'] == 12.5 + assert payload['cost_cents'] == 3.2 + assert payload['checkpoint'] == checkpoint + + +def test_tick_without_provider_keeps_legacy_payload() -> None: + audit = AuditLogger() + beat = Heartbeat(audit, 'run-legacy', interval_seconds=5.0) + beat.tick() + + (payload,) = _heartbeat_payloads(audit) + assert payload == {'tick': 1, 'interval_seconds': 5.0} + + +def test_tick_omits_fields_the_provider_does_not_return() -> None: + audit = AuditLogger() + beat = Heartbeat( + audit, + 'run-partial', + interval_seconds=5.0, + progress=lambda: {'elapsed_seconds': 1.0}, + ) + beat.tick() + + (payload,) = _heartbeat_payloads(audit) + assert payload == {'tick': 1, 'interval_seconds': 5.0, 'elapsed_seconds': 1.0} + + +def test_provider_cannot_overwrite_tick_or_inject_unknown_keys() -> None: + audit = AuditLogger() + beat = Heartbeat( + audit, + 'run-guard', + interval_seconds=5.0, + progress=lambda: {'tick': 999, 'run_id': 'other', 'cost_cents': 1.5}, + ) + beat.tick() + + (payload,) = _heartbeat_payloads(audit) + assert payload['tick'] == 1 + assert 'run_id' not in payload + assert payload['cost_cents'] == 1.5 + + +def test_failing_provider_does_not_stop_the_tick() -> None: + audit = AuditLogger() + + def broken() -> Optional[dict[str, Any]]: + raise RuntimeError('state unavailable') + + beat = Heartbeat( + audit, + 'run-broken', + interval_seconds=5.0, + progress=broken, + ) + beat.tick() + + (payload,) = _heartbeat_payloads(audit) + assert payload['tick'] == 1 + assert payload['progress_unavailable'] is True + + +def test_real_run_emits_heartbeat_with_current_tool_and_elapsed( + chat_agent_config: ChatAgentConfig, + tmp_run_store: RunStore, +) -> None: + (chat_agent_config.root / 'note.txt').write_text('hello', encoding='utf-8') + adapter = FakeAdapter( + [ + '{"type":"tool","tool_name":"workspace_read_file","arguments":{"path":"note.txt"},"call_id":"read-hb"}', + '{"type":"final","content":"done"}', + ], + before_each=lambda: time.sleep(0.05), + ) + audit = tmp_run_store.audit_logger() + + result = run_chat_agent( + ChatAgentConfig.from_root(chat_agent_config.root, heartbeat_seconds=0.02), + 'read the note', + adapter=adapter, + audit=audit, + ) + tmp_run_store.logger_for_result(result, audit) + + assert result.status == 'completed' + beats = [e.payload for e in audit.events if e.event_type == 'heartbeat'] + with_tool = [b for b in beats if b.get('tool') == 'workspace_read_file'] + assert with_tool, 'expected at least one heartbeat to name the running tool' + assert all(b['elapsed_seconds'] >= 0 for b in with_tool) + assert all(b['cost_cents'] >= 0 for b in with_tool) diff --git a/tests/test_subagent_batch_cancel_token.py b/tests/test_subagent_batch_cancel_token.py new file mode 100644 index 00000000..9da5f7fc --- /dev/null +++ b/tests/test_subagent_batch_cancel_token.py @@ -0,0 +1,182 @@ +# test-type: behavior +"""A batch deadline must stop running children via the cooperative cancel token. + +Covers G-P1-5: children that outlive the batch deadline keep executing tool +calls unless the batch sets a cancel token they check. These tests drive a real +child AgentRunner (no mocked run_subagent) with a stub LLM that always requests +a slow read-only tool, and count how many tool calls actually execute. +""" + +from __future__ import annotations + +import threading +import time +from pathlib import Path +from tempfile import TemporaryDirectory + +from teaagent.chat_agent import ChatAgentConfig +from teaagent.llm import LLMResponse +from teaagent.run_store import RunStore +from teaagent.subagents import SubagentManager +from teaagent.subagents._tools import register_subagent_tools +from teaagent.types import ToolAnnotations, ToolRegistry + +_TOOL_SLEEP_SECONDS = 0.4 +_BATCH_TIMEOUT_SECONDS = 1 +_POST_RETURN_WAIT_SECONDS = 1.5 + + +class _CountingProbe: + """Thread-safe counter for executed tool calls, with a slow handler.""" + + def __init__(self) -> None: + self._lock = threading.Lock() + self._count = 0 + + def __call__(self, _args: dict) -> dict: + with self._lock: + self._count += 1 + time.sleep(_TOOL_SLEEP_SECONDS) + return {'ok': True} + + def value(self) -> int: + with self._lock: + return self._count + + +class _AlwaysToolAdapter: + """Stub LLM that requests the slow probe tool on every iteration.""" + + provider = 'stub' + + def __init__(self) -> None: + self._lock = threading.Lock() + self._calls = 0 + + def complete(self, request: object) -> LLMResponse: + with self._lock: + self._calls += 1 + call_id = f'c{self._calls}' + return LLMResponse( + provider='stub', + model='stub', + content=( + '{"type":"tool","tool_name":"slow_probe",' + f'"arguments":{{}},"call_id":"{call_id}"}}' + ), + ) + + +def _setup( + root: Path, parent_cancel: threading.Event | None +) -> tuple[ToolRegistry, SubagentManager, ChatAgentConfig, _CountingProbe]: + (root / '.teaagent').mkdir(exist_ok=True) + probe = _CountingProbe() + registry = ToolRegistry() + registry.register( + name='slow_probe', + description='slow read-only probe', + input_schema={'type': 'object', 'properties': {}}, + output_schema={'type': 'object', 'additionalProperties': True}, + annotations=ToolAnnotations(read_only=True), + handler=probe, + ) + config = ChatAgentConfig(root=root, max_iterations=200, cancel_token=parent_cancel) + manager = SubagentManager( + root=root, parent_config=config, parent_adapter=_AlwaysToolAdapter() + ) + return registry, manager, config, probe + + +def _run_batch( + registry: ToolRegistry, + manager: SubagentManager, + config: ChatAgentConfig, + tasks: list[dict], + timeout_seconds: int, +) -> dict: + # register_subagent_tools binds ``registry`` as the children's source registry, + # so the batch tool must be registered on the same registry the probe lives in. + register_subagent_tools( + registry, + adapter=manager._parent_adapter, + config=config, + depth=0, + manager=manager, + ) + return registry.execute( + 'subagent_batch', + { + 'tasks': tasks, + 'max_workers': len(tasks), + 'timeout_seconds': timeout_seconds, + }, + ) + + +def _loop_task() -> dict: + return { + 'task': 'loop forever', + 'isolation': 'shared', + 'max_iterations': 200, + 'max_tool_calls': 200, + } + + +def test_batch_deadline_stops_running_child_tool_loop() -> None: + """After the batch returns at its deadline, a running child may finish at most + the one tool call it had already dispatched; it must not start another. + + Bound: tool executions after batch return <= 1 (the single in-flight call + that passed the pre-dispatch cancel check before the batch token was set). + """ + with TemporaryDirectory() as td: + root = Path(td) + registry, manager, config, probe = _setup(root, parent_cancel=None) + result = _run_batch( + registry, manager, config, [_loop_task()], _BATCH_TIMEOUT_SECONDS + ) + count_at_return = probe.value() + time.sleep(_POST_RETURN_WAIT_SECONDS) + count_after_wait = probe.value() + + assert result['timed_out'] == 1, result + assert count_at_return >= 1, 'child never executed a tool before the deadline' + delta = count_after_wait - count_at_return + assert delta <= 1, ( + f'child executed {delta} tool calls after the batch deadline ' + f'(bound: <= 1 in-flight call); at return={count_at_return}, ' + f'after wait={count_after_wait}' + ) + + store = RunStore(root) + cancelled_runs = [] + for summary in store.list_runs(limit=None): + events = store.show_run(summary.run_id) + if any( + e.get('event_type') == 'run_failed' and 'cancel' in str(e).lower() + for e in events + ): + cancelled_runs.append(summary.run_id) + assert cancelled_runs, 'child cancellation was not recorded in the audit log' + + +def test_parent_cancel_mid_batch_stops_children() -> None: + parent_cancel = threading.Event() + with TemporaryDirectory() as td: + root = Path(td) + registry, manager, config, probe = _setup(root, parent_cancel=parent_cancel) + timer = threading.Timer(0.6, parent_cancel.set) + timer.start() + started = time.monotonic() + result = _run_batch(registry, manager, config, [_loop_task(), _loop_task()], 10) + elapsed = time.monotonic() - started + timer.cancel() + count_at_return = probe.value() + time.sleep(0.8) + + assert elapsed < 5, ( + f'batch waited {elapsed:.2f}s; parent cancel did not propagate' + ) + assert all(r.get('status') != 'completed' for r in result['results']), result + assert probe.value() - count_at_return <= 1