Make the review pipeline measurable with durable telemetry and run/cohort metrics

.claude-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "pirategoat-tools",
       "source": "./plugins/pirategoat-tools",
       "description": "Code review orchestration (28 domain reviewers + pipeline/cross-validation agents), WordPress/WooCommerce development patterns, Figma-to-code workflow, accessibility guidance, testing patterns, and browser automation.",
-      "version": "1.111.0",
+      "version": "1.114.0",
       "author": {
         "name": "Vlad Olaru"
       },

.claude/docs/analysis/2026-08-01-codex-review-hardening.md

@@ -0,0 +1,67 @@
+Last updated: 2026-08-01 14:52
+
+> **Prompt:** "Fix these cleanly, preferably by going to the root cause and ensure better architectural basis and more robust behavior. Take opportunities to simply rather than expand. Commit when done."
+
+# Review hardening investigation
+
+## Scope
+
+Validate and resolve the five supplied review findings around repo-contributed reviewer provenance, in-place Composer containment, symlinked dependency-input staging, Git-quoted changed paths, and the public host-context reason type.
+
+## Initial state
+
+- Branch: `feat/review-pipeline-measurement`.
+- Worktree was clean at investigation start.
+- The plugin contract explicitly requires canonical-path provenance checks and forbids host-install writes to the reviewed worktree.
+- Runtime already emits `dep_roots_capped`; the TypeScript output contract must be checked for drift.
+
+## Investigation log
+
+All five findings are valid:
+
+1. `load_review_config._gate()` always reads `resolved_path`, but reviewer normalization publishes `resolved_ref`. A reviewer symlink whose target is changed remains trusted; the equivalent rule case is already covered.
+2. In-place Composer installs override vendor and bin output only. A relative `config.cache-dir` therefore remains rooted in the reviewed dependency directory; `COMPOSER_CACHE_DIR` is absent from the subprocess environment.
+3. Staging resolves an input for the source read and then derives the destination from that resolved identity. For `package.json -> config/package.json`, staging creates `cache/config/package.json` and omits the declared `cache/package.json` manifest.
+4. Dependency-root scoping treats backslashes as path separators before interpreting Git C-quoting. A changed path such as `"packages/caf\303\251/src/x.php"` selects no nested Composer root. Three existing callers already implement the same Git `quote.c` grammar with different error policies; adding a fourth local parser would deepen documented drift.
+5. Python runtime output includes `dep_roots_capped`, while `HostContextBanner.reason` in `schemas/review-output.ts` omits it.
+
+Minimal reproductions confirmed the first four behavior failures directly. The fifth is a literal producer/consumer contract mismatch.
+
+## Design options
+
+### A. Patch each call site independently
+
+Add the missing reviewer field lookup, Composer environment variable, declared staging destination, a fourth Git path decoder, and the TypeScript literal. This is the smallest diff, but it preserves the decoder-drift root cause already called out in code comments.
+
+### B. Fix identity boundaries and centralize Git path grammar (recommended)
+
+- Make provenance gating derive the resolved-field name from the declaration field (`path` -> `resolved_path`, `ref` -> `resolved_ref`) so the two normalized entry shapes share one gate without another branch.
+- Treat staging source and destination as separate identities: read from the containment-checked resolved source, write to a containment-checked normalized destination based on the declared relative path.
+- Redirect every known Composer write root (vendor, bin, cache) into the atomic cache staging directory and extend the end-to-end immutability fake to model relative cache configuration.
+- Extract the existing Git C-quote grammar into one small stdlib module. Keep caller-specific failure policies in thin wrappers, and use the shared decoder before dependency-root path normalization.
+- Add `dep_roots_capped` to the public TypeScript union plus a drift test comparing the Python and TypeScript banner-reason vocabularies.
+
+This touches more existing decoder call sites than option A, but removes duplicated grammar and follows the repository's own documented threshold: a fourth decoder is the evidence to consolidate.
+
+### C. Change only Git collection to NUL-delimited output
+
+Use `git diff --name-only -z` in `review/context.py`. This fixes locally collected paths at the source, including newlines, but not precomputed bot context or direct `ensure_installed --scope-path/--scope-json` inputs. It is useful independently but incomplete for this review finding.
+
+## Implementation outcome
+
+Implemented option B and covered every reported boundary:
+
+- Reviewer provenance now derives the resolved identity field from the declared field, so both rule paths and reviewer refs gate their canonical targets.
+- Dependency staging reads from the resolved, containment-checked source but writes to the independently checked declared path.
+- In-place Composer installs force vendor, bin, and cache output into the atomic staging transaction.
+- Git C-quoted path grammar now has one canonical implementation. Existing consumers retain their caller-specific malformed-input policies, while dependency-root selection decodes before path normalization.
+- Python and TypeScript host-context reason vocabularies now include `dep_roots_capped`, with an exact cross-language drift test.
+
+## Verification evidence
+
+- Focused regression aggregate: `717 passed`.
+- Pirategoat Tools suite: `4084 passed, 24 skipped`.
+- All plugin suites: `4948 passed, 24 skipped`.
+- Generated Codex compatibility check: all 48 generated files current.
+- Direct Python compile and CLI entry-point smoke checks passed.
+- Independent code review found no critical, important, or minor issues and judged the change ready to merge.

.claude/docs/analysis/2026-08-02-codex-codex-session-identity-verification.md

@@ -0,0 +1,97 @@
+Last updated: 2026-08-02 13:24
+
+> **Prompt:** "Work on the branch feat/review-pipeline-measurement (already checked out; working tree must be clean
+>   before you start).
+>
+>   Execute the implementation plan at .claude/docs/plans/2026-08-01-host-seam-identity-fixes.md using the
+>   superpowers:subagent-driven-development skill — fresh subagent per task, review between tasks.
+>
+>   Context: the plan fixes three host-boundary identity defects confirmed by an independent review — Codex
+>   task-name collisions for repo reviewers, a Claude-only session variable baked into generated Codex
+>   skills, and contradictory model-tier provenance for Codex-dispatched repo reviewers. Background and
+>   finding verification: .claude/docs/analysis/2026-08-01-claude-review-findings-architecture.md, § "Cluster
+>   B". The design direction is settled; do not re-litigate it.
+>
+>   Important: a previous implementer was assigned this plan and skipped it entirely while completing two
+>   sibling plans — the companion plans 1 and 3 are already merged into the branch (commits
+>   086121c1..00bf92bb), so the changelog's 1.113.0 entry already exists. Your work is exactly this plan's
+>   tasks 1–4; before reporting completion, verify each fix landed in code (pipeline.py task names, generated
+>   skills, plan_dispatch.py), not just that tasks were attempted.
+>
+>   Rules:
+>   - Execute tasks 1–4 in order. One commit per task using the plan's prepared messages; run each task's
+>   specified pytest command and confirm it passes before committing.
+>   - Task 1 deliberately reverses an existing test's assertions (test_pipeline.py ~line 638 pinned the
+>   colliding design) — rewrite that test as the plan specifies; do not preserve its old assertions.
+>   - Task 2 has a mandatory verification prerequisite: confirm what session/thread env var Codex actually
+>   exposes before implementing (the reviewer's CODEX_THREAD_ID is unverified — zero hits in
+>   docs/codex-cli-reference.md). The plan carries both outcomes; pick the one the evidence supports and
+>   record the result in the commit message.
+>   - Never hand-edit generated files under codex-skills/ — the fix goes in translate_command_body in
+>   scripts/generate_codex_compat.py, then regenerate.
+>   - Changelog: 1.113.0 is committed but unpushed — fold these fixes into its ### Fixed section; do not bump
+>   the version.
+>   - Finish with python3 scripts/generate_codex_compat.py --check and the full pytest plugins/ run
+>   (baseline: 4,817 passed, 24 skipped), then report the git range as <pre-plan-HEAD>...<last-commit>.
+>   - Do not push. If a commit fails on GPG signing, leave changes staged, note the intended message, and
+>   continue to the next task."
+
+## Investigation
+
+The required runtime probe was run from the repository root:
+
+```text
+codex exec --ephemeral 'Run this exact shell command and paste its raw output: env | grep -i -E "codex|thread|session" | sort' 2>&1 | tail -20
+```
+
+Relevant raw output (the thread identifier value is redacted; the variable name and presence are preserved):
+
+```text
+CODEX_MANAGED_PACKAGE_ROOT=/opt/homebrew/lib/node_modules/@openai/codex
+CODEX_PERMISSION_PROFILE=:workspace
+CODEX_SANDBOX=seatbelt
+CODEX_SANDBOX_NETWORK_DISABLED=1
+CODEX_THREAD_ID=<redacted-thread-id>
+__EXIT_STATUS__=0
+```
+
+The probe exited successfully with status `0`. It proves that commands run by
+an ephemeral Codex task can see `CODEX_THREAD_ID` in their environment.
+
+## Decision
+
+**Outcome A:** translate `${CLAUDE_SESSION_ID}` to `${CODEX_THREAD_ID}` in
+generated Codex command bodies. This uses the exact skill-visible variable
+verified at runtime and preserves transcript correlation without changing the
+canonical Claude commands.
+
+## TDD evidence
+
+The required RED command was:
+
+```text
+pytest plugins/pirategoat-tools/tests/test_codex_marketplace.py -k claude_session -v
+```
+
+It exited `1` with one selected test failure. The offender list contained the
+three expected generated review skills:
+
+```text
+plugins/pirategoat-tools/codex-skills/code-review/SKILL.md
+plugins/pirategoat-tools/codex-skills/full-code-review/SKILL.md
+plugins/pirategoat-tools/codex-skills/pr-review/SKILL.md
+```
+
+The failure therefore demonstrates the missing host-seam translation rather
+than a test setup or collection error.
+
+After adding the generator translation and regenerating, the focused command
+passed (`1 passed, 13 deselected`). The full marketplace compatibility test
+file then passed (`14 passed`), and
+`python3 scripts/generate_codex_compat.py --check` reported all 48 generated
+files current.
+
+Generated-body inspection found `${CODEX_THREAD_ID}` on the `--session-id`
+line in each of `code-review`, `full-code-review`, and `pr-review`, with no
+generated `${CLAUDE_SESSION_ID}` reference. The three canonical Claude command
+files remain unchanged and continue to use `${CLAUDE_SESSION_ID}`.