Skip to content

VERIFY.md

Latest commit

 

History

History
84 lines (64 loc) · 3.85 KB

VERIFY.md

File metadata and controls

84 lines (64 loc) · 3.85 KB
Raw

VERIFY.md — TeaPrompt verification control surface

Generated by the verification-map-generator domain pack (dogfooding run, 2026-09-23; corrected 2026-09-24 after independent review; dependency setup corrected 2026-10-10). Audience: a fresh agent with no conversation history. The gate and doctor commands require the developer environment below.

What this repo is

A Markdown prompt/skill library plus a Python governance gate suite. There is no server and no build artifact: the "product" is the prompt corpus and its validators. Verification = running the gate suite and checking registry invariants.

Launch

cd "$(git rev-parse --show-toplevel)"   # repo root; all commands run from here
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r reflective-prompt-library/requirements-dev.txt
make all                                # full gate: pytest + 8 validator scripts + 3 route evals

Expected: exit 0. Total wall time ~15–30 s.

Doctor (fast smoke, ~2 s)

python3 reflective-prompt-library/plans/validate_links.py        # expect: "All validations passed!"
python3 reflective-prompt-library/plans/validate_governance.py   # expect: "valid governance metadata"

If doctor fails, stop: the repo is unhealthy and a full sweep wastes time.

Drive

Per-feature commands live in features/:

Area File Surface
Governance test suite features/test-suite.md python3 -m pytest reflective-prompt-library/plans/tests/
Standalone validators features/validators.md 7 plans/validate_*.py + lint_skills.py (8 scripts)
Route paraphrase evals features/route-evals.md route_paraphrase_eval.py over 3 YAML fixtures
Skill registry features/registry.md CORE_SKILLS (9) / DOMAIN_PACK_SKILLS (10) cardinality

Environment quirks (verbatim)

  • route_paraphrase_eval.py writes plans/route-00X-results.json on every run. These files are gitignored and never appear in git status.
  • lint_skills.py exits 0 when only warnings are reported. The 2026-10-07 drive reported eight long-skill warnings across six packs, not one. Preserve the full warning output; new or changed warnings remain findings to triage. The R10 acceptance covers agent-governance-scaffold body length, not a blanket waiver for other warnings. Any lint error is a failure.
  • Dated supersession (2026-10-12): approved checkpoint reference factoring reduced the observed set to five warnings in three main contracts: arm-blinded-eval-harness, router-trace-linter, and governed-delivery. The current inventory and disposition preserves the earlier baseline and unchanged thresholds; it is not a waiver.
  • make validate runs the route evals; make test runs pytest. make all runs both.
  • Repository tooling requires Python 3.9+ and the development requirements (pytest plus the validator dependencies). Activate the environment before doctor, Make or pre-commit: the language: system hook uses python3 on PATH and does not install dependencies.

Failure classification

Class Meaning Repair
Product regression Repo behavior changed for the worse (validator fails, test fails, registry cardinality changed) Report; never edit map/spec/tests to match
Doc drift This map no longer matches a healthy repo Update the doc
Spec/oracle error acceptance.yaml expectation is wrong Spec is read-only by convention; propose change for human review
Harness failure Python/pytest/toolchain broke, not the repo Fix the harness

Evidence

Keep command output. For a sweep report, record: commit, date, per-feature pass/fail, and classification of every failure.

Cleanup

None. Eval runs write only gitignored route-00X-results.json files.