Repository navigation
[docs] Five documented test counts, three different values, none correct #170
Description
Context
The repository's stated discipline is that a number written into a document must be computable by
one command — CODEBASE.md:547 restates it as "the README badge carries the test count and is part
of the same measurement discipline". Five places carry a test count and they disagree with each
other and with the tree.
Measured today at 7e6b017:
find . -name '*_test.go' -not -path './tuiprobe/*' -print0 \
| xargs -0 grep -h '^func Test' | wc -l # 749 (main module; what `go test ./...` runs)
grep -rn '^func Test' --include=*_test.go . | wc -l # 848 (includes tuiprobe/, a nested module)
grep -rn '^func Fuzz' --include=*_test.go . | wc -l # 10| Location | Says | Measured |
|---|---|---|
README.md:15 (badge) |
tests-710 |
749 |
README.md:136 (changelog entry) |
"359 tests total" | historical for cbd6db3; correct to leave as history if it is labelled as such |
CODEBASE.md:3 (preamble) |
"771 test functions + 10 fuzz targets" | 749 + 10 |
CODEBASE.md:547 |
"771 test functions + 10 fuzz targets" | 749 + 10 |
CODEBASE.md:582 (comment beside its own command) |
"# 710 test functions" | 749 |
So CODEBASE.md disagrees with the command printed directly above the claim it annotates.
What to build
- Correct the badge and both
CODEBASE.mdprose figures to whatgo test ./...actually runs
(749), or state explicitly that the figure counts the nested module too (848) — pick one and say
which. - Correct the comment at
CODEBASE.md:582so the command and its expected output agree. - Leave
README.md:136's historical figure alone if it is clearly scoped to that changelog entry.
Acceptance
- Every test-count figure in tracked markdown is reproduced by one command pasted next to it.
CODEBASE.md:582's comment equals the output of the command it annotates, on a clean checkout.- No two tracked documents state the same measurement with different values.
Activity
docs(roadmap): register the verified missing-capability inventory
docs(roadmap): register the verified missing-capability inventory
yusiwen commentedon Oct 10, 2026
A measurement to fold in when this is fixed: the number moved again.
Measured at
d5a625a(after PR #175, which addedagent/usage_test.goandtui/usage_test.go):So: 767 test functions in the main module, 866 repository-wide. The roadmap's
7e6b017baseline figures (749 and 848) stay true of that baseline, but any count taken from the current tree
has to be 767/866 — the
771inCODEBASE.mdand the710in the README badge were already wrongand are now 57 and 156 apart from the tree respectively.
yusiwen commentedon Oct 13, 2026
Fixed in PR #213, rebase-merged to
masteras3c2b64f. The badge is removed rather than re-badged, on the user's decision.The five places, all corrected to one measurement (857 test functions in the main module + 10 fuzz targets, as of
1f101ec):README.mdbadgetests-710README.mdproseCODEBASE.mdpreambleCODEBASE.mdTestingCODEBASE.mdbeside its own command# 710 test functions# 857 test functions (main module)Your third bullet needed a decision, not a number. The command in "How these numbers are measured" was
grep -rn '^func Test' --include=*_test.go . | wc -l, which counts the nestedtuiprobemodule too — 956 today — while the prose figure counted the main module. So "correct the comment so the command and its expected output agree" could be satisfied two ways; I chose the main-module counting, since that is whatgo test ./... -count=1runs, and changed the command to exclude the nested module so the command, its comment and the prose all state 857.Why removal over updating. The badge had no code path keeping it honest: it read 564 while the document said 667, then 710 while the tree had 857. Updating it once more restarts that clock. The paragraph that documented its drift now records why it is gone, and the two sentences that leaned on it are gone too — a summary sentence is exactly what rots when the thing it summarises is deleted, and that is what your issue was really about.
Six figures deliberately left as history:
CHANGELOG.md's 27 goldens and 710 tests (entries scoped to their commit),docs/roadmap.md's 749 (the7e6b017baseline every roadmap entry measures against), and three insidetuiprobe/'s own released records. Your body said the same aboutREADME.md's historical figure, so the same reasoning applies to them — a measurement that was true when taken is not a defect.Audit after the change: 0 broken relative links of 28 checked;
tests-710andbadge/testsappear 0 times inREADME.mdandCODEBASE.md; 0 live count claims mismatched against the tree; 6 historical figures left with their reason recorded in the PR body. One caveat worth stating: the counts are correct at1f101ecand will drift again as the suite grows — what the PR fixes is that each one now sits beside the command that produces it, which is the only thing that makes the next correction mechanical.