Repository navigation
Slim the Lean capture code and tests #257 added, without changing what they pin - #307
Conversation
#257 pinned each capture race in its own test function: its own tree, its own checkpoint or os patch, and the same capture-once or caught-and-named assertions. Most of them share one of two shapes, so they are now two tables: - test_change_without_lean_inputs_is_captured_once: 18 changes that touch no Lean input and must be captured in one attempt (rebuilt, cleaned or removed sites, atomic saves, lock links, republished or removed skeleton output, recloned or removed checkouts, entries that vanish before their lookup or read). - test_lean_change_is_caught_and_named: 19 Lean changes that must fail the capture and name the path (Lean files appearing anywhere, entries seen only by a folder's end-of-scan relist, folders moved or copied during their parent's scan, markers exposing or hiding Lean files). One helper, _at, installs every hook these races use (the checkpoint before verification, the checkpoint after a folder is listed, a folder's end-of-scan relist, the first open of a name) and replaces _before_relist_of and _on_open_after_checkpoint. The three tests of the "kept changing" message are one parametrized test, and the marker test in test_tree_snapshot_limits.py is a three-row table instead of one test that reruns itself. Removed as duplicates: - the zz/site copies of the rebuilt-folder cases: the same code path and the same mutant kills as the Aout/site ones; - the per-case check that a rebuilt site stays out of TreeSnapshot.directories, which test_unrelated_folders_do_not_change_tree_revisions pins through the revision; - the recapture after a Lean file appears in an unrelated folder, which only shows that a .lean file there is indexed. Races with several steps now also assert that the final verification is never reached, which is what "only the relist sees it" means. The 68 selected-mode mutants collected over #257's review rounds are killed the same way before and after (55 killed), and every removed case maps to a remaining one that fails on main before #257.
Selected-mode verification had its own copy of the end-of-scan relist, verify_selected_directory: list the folder through _relist_directory, refuse a marker that now hides recorded names, require every recorded name, and walk every other name with _require_no_inputs. It differed from _recheck_scanned_directory only in having no identities for unrecorded names and in verifying each recorded name where the listing has it. _recheck_scanned_directory now takes verify_kept, and verification calls it with no unkept identities. The marker check is the same: the scan's relist raises only when the folder kept names, and verification reaches only recorded folders, which are the root (never checked for a marker) or folders that kept names. The strict and selected loops both verified a recorded name through the same lookup and the verify_entry/verify_child_directory pair; that is one verify_recorded closure again, as before #257. Checks run in the same order and name the same paths: the 37 filesystem-history probes print the same output, listing counts included, and the 68 selected-mode mutants, adapted to the merged function, are killed as before.
Since #257 every Lean capture runs in selected-entry mode. The Lean selection's include admits only .lean entries outside the excluded folders, the scan records no marker or unselected entry, and it records only the root and the folders that hold a recorded entry. So in _indexed_source_snapshot the suffix and exclusion filters on captured files drop nothing, the suffix filter on unsupported entries drops nothing, and _lean_generation_revision rebuilds a TreeSnapshot equal to the one it was given. These predate #257, when the Lean include also captured manifest.json files and the index re-derived skeleton markers. They go, with BoundProjectSources.excluded, whose only reader they were; the Lean selection stays the one place that decides what a Lean input is. To check the claim, a throwaway checkout of main asserted, on every Lean index build of the full suite, that the three filters dropped nothing and that _lean_generation_revision equalled snapshot.generation_revision: 779 builds, no difference. The retry loop in _open_stable_project_sources now closes a failed binding in one handler instead of three.
The scan's docstring repeated TreeSelection's description of selected mode, _relist_directory's and _require_no_inputs's spelled out cases their first sentence already covers, and the comment on the Lean selection listed the folders the README and TreeSelection already name. Each now says what its function adds. No code changes.
Passing verify_recorded to _recheck_scanned_directory through a lambda added a fourth Python frame per folder level to selected-mode verification (verify_directory, _recheck_scanned_directory, the lambda, verify_recorded). Lean capture sets no max_depth, so the recursion limit decides how deep a tree it accepts: with Python's default limit, main captures a single-chain Lean tree up to 328 folders deep, and the lambda lowered that to 246, failing deeper trees with "directory tree is nested too deeply to capture". A functools.partial adds no Python frame, which restores main's three frames and main's 328 on Python 3.13 and 3.10. A new test counts the Python frames verification adds per folder level in a Lean capture and pins it at three. It fails with the lambda (four) and passes on main.
|
Verdict: changes requested. One test helper needs a small fix, and the description needs two corrections. This is an adversarial review of head 01ba073: one reviewer checked whether the code still behaves the same, another whether the slimmed tests still pin what #257's did, and a verifier tried to refute each finding. The code change preserves behaviour. What was checked
Should fix
Optional
Posted by PR swarm: PR Swarm Lead |
Review of #307 found that the `functools.partial` keeps three Python frames per folder level but re-enters the interpreter from C at each level, which spends CPython's separate C recursion budget: with a raised recursion limit, or little C budget left at the call site, a capture failed where main's succeeded. `_recheck_scanned_directory` now calls `verify_kept(descriptor, relative, name)`, and verification passes `verify_recorded` itself: three Python frames per level and no C-level call. Its docstring now says that the relist bounds the undetected window at the end of a scan, and that verification passes an empty `unkept`. The scan's docstring again says what the default mode records.
Since verification relists through `_recheck_scanned_directory` too,
`_at("relist F")` fired at verification whenever the end-of-scan relist
did not run, so the two captured-once rows that use it passed even with
that relist removed. The hook now fires only on the scan's call, which
passes no `verify_kept`, and each row already asserts that its hook ran.
The kept-changing test skips without descriptor capture instead of
guarding its message assertion, which could never be reached there,
since its setup already needs descriptor capture. A new test checks that
a selected-mode capture releases its selection without the cyclic
collector, since selected-mode verification now relists through a
callback.
|
Update: ready to merge once it is out of draft. At the user's request I pushed the fixes from my review above as two commits on top of 01ba073. Head 24d6ea4 passes all 11 checks, real Lean included, and merges cleanly with main.
A scratch run on 24d6ea4 killed three mutants: skipping the end-of-scan relist, verification skipping kept names, and keeping the verification cycle. Not changed: the frame test still pins exactly three frames per level. That is the author's choice, and it keeps the comment honest. Posted by PR swarm: PR Swarm Lead |
|
Post-merge adversarial review: follow-up needed (reviewed merged 54127e9) Runtime behavior of the merged code matches 774adc0 on everything we probed. The follow-up is one missing test, for a regression this PR's own history already hit once. What was checked, across five lenses:
Confirmed finding (minor):
Refuted: the stale #90 overlap note. The PR body now says #90 already conflicts with main, and #90 conflicts with 774adc0 as well. Optional nits:
|
Resolves the conflicts with #212, #300, #301, #307 and #314, and adds the branch's new template to the list #317 reads. The other merged PRs merge as is. - lean: capture and _indexed_source_snapshot take names= and no longer the excluded folders #307 removed; the retry loop keeps #307's single handler, and _lean_generation_revision goes as #307 removed it. A file the name filter admits is scanned with #212's #exit-aware line count and records names other files also declare. A file it skips keeps the plain line count, which no declaration is sized against, so line_counts still covers every file. - lean, render: SourceLinker.url, _SourceBase.href and _vault_source_link name the spelling Git committed (#301) and quote the ref as one path segment. - graph: take both sides' imports. The body is masked once with #212's mask_code_and_comments and read for both links and the summary, so a block the page shows as indented code is neither a dependency nor summary text. - runtime: drop the mathlib_file check, which #314 moved into the graph loader, and keep the catalog checks beside it. - README: keep the dashboard and implementation-note sentences and the autoform-publication/v3 paragraph around #300's rules for code links and links into the blueprint. - scaffold: #317 scaffolds only the paths in _TEMPLATE_PATHS, so list hooks/title_only_search.py; without it new projects lose the hook their mkdocs.yml names. - tests: test_graph takes both imports. New: the summary skips a block shown as indented code, a filtered index has a line count for every file, and links into the blueprint quote the ref and name the committed spelling.
facebookresearch#307 passes verify_recorded to _recheck_scanned_directory as a callback. Its review history found two ways a wrapper costs capture depth: a lambda adds a Python frame per folder level (pinned by test_selected_verification_recurses_in_three_frames_per_folder), and a C-implemented callable such as functools.partial adds no Python frame but spends CPython's separate C recursion budget per level, so a deep Lean tree fails with "nested too deeply" where main captured it. fd731bd removed the partial, but nothing caught putting it back: the frame count cannot see it. The new test records the type of every verify_kept callback in a Lean capture and requires a plain Python function. It fails with functools.partial(verify_recorded) restored and passes on main.
Nothing in autoform_cli or servers uses these parts of the capture: - TreeSelection.placeholder and TreeSnapshot.placeholders. No production selection has ever passed placeholder=. facebookresearch#307 deleted lean.py's _lean_generation_revision, the last reader outside this module, and the previous commit deleted the materialization, the last one inside it besides revision. _selected goes with them; _require_no_inputs asks include directly. - TreeSelection.byte_limit, and file_read_limit's check of a selected limit. facebookresearch#257 removed the last production selection that set it. file_read_limit's cap now only comes from OpaqueDirectoryMarker.max_bytes, which the marker validates, so its parameter is max_bytes, defaulting to None, instead of selected_limit. - BoundDirectoryTree.descriptor, which nothing reads, and capture's selection override with _capture_locked: lean.py, render.py and library/verify.py call capture() with no arguments. - The os.listdir fallback: _DESCRIPTOR_SCANDIR_SUPPORTED, _requires_bounded_enumeration and _directory_names_match's limits parameter. Descriptor capture now checks os.scandir in os.supports_fd, the only listing call left, instead of os.listdir. CPython adds both there together, under HAVE_FDOPENDIR, in every version from 3.10, so the check admits the same platforms, and only a test that patched the flag reached the fallback. Without it, _capture_path_names repeated _capture_directory_names, which now takes a descriptor or a path, and add_entries becomes add_entry, since every call counted one entry. - The RLock around the binding's methods. Every caller opens, uses and closes its binding on one thread. - bind_directory_tree's require_descriptor keyword, which facebookresearch#235 left with no caller. BoundDirectoryTree keeps its own: lean.py passes it, and its "safe directory traversal is unavailable on this platform" error is CLI output. - TreeSnapshot.root_identity, which no revision hashes and which facebookresearch#307 left with no reader; the root's identity is already in identities. The never-read BoundDirectoryTree.expected_identity attribute goes too; the parameter stays, since render passes it. No production code on main has ever set TreeCaptureLimits.max_depth or max_file_bytes apart from the materialization the previous commit deleted: render's setters were added and removed on facebookresearch#165's branch before it merged. library/verify.py sets only max_entries and max_total_bytes. Both go, with the depth argument that the scan, the relists, the marker check and the portable walk passed along only for the max_depth check. max_total_bytes still caps each file read, and the recursion limit still bounds depth. Six tests covered only these and go: capture and close serialization, the invalid selection byte limit, the depth and file-byte boundaries, the composed byte limits, and the listdir fallback with its helper. The entry-limit test counts two ignored files instead of an ignored file and a placeholder, and the descriptor and portable comparison drops its placeholder and keeps max_entries at its boundary. The overflow test now uses max_total_bytes and is renamed for it. The oversized-file test goes, since without max_file_bytes it repeated the total-limit test's check that a file over the budget is never read. The invalid-limit test checks its bool row on max_entries and its float row on max_total_bytes, the portable relist test patches the one remaining lister, and no comment names max_depth any more. No production snapshot ever held a placeholder, so the dropped digest loop hashed nothing, and root_identity was never hashed: no digest or revision changes. The error messages that go could not be reached from any command, so no JSON output or CLI behavior changes.
Summary
The slimming follow-up asked for when #257 merged. Net -462 lines (+660/-1122) in five commits, plus two review-fix commits (see Review fixes below); behavior is unchanged.
tests/test_lean_sources.py+553/-899,tests/test_tree_snapshot_limits.py+29/-35). Prove Lean capture stability only for Lean inputs, and name what changed #257 gave each capture race its own test function; most had one of two shapes, so they are now two tables:test_change_without_lean_inputs_is_captured_once: 18 changes that touch no Lean input and must be captured in one attempt (rebuilt or removed sites, atomic saves, lock links, republished skeleton output, recloned checkouts, entries that vanish before their lookup or read).test_lean_change_is_caught_and_named: 19 Lean changes that must fail the capture and name the path._at, installs every hook these races use, replacing two. The "kept changing" message tests are one parametrized test.zz/site, clean onAout/site, vanish onzz/site; each shape keeps one folder, and both folders take the same code path and kill the same mutants). Also removed: a per-case directory check, whichtests/test_tree_snapshot_limits.py::test_selected_capture_ignores_unselected_churnpins (directories == ("",)for a folder holding only folders); and a recapture that only showed a Lean file gets indexed.test_selected_verification_recurses_in_three_frames_per_folder, which pins the Python frames selected-mode verification spends per folder level (see below). It fails with four and passes on main._tree_snapshot.py(+60/-107). Selected-mode verification had its own copy of the end-of-scan relist; it now reuses_recheck_scanned_directorythrough averify_keptcallback. The callback isverify_recordeditself, called asverify_kept(descriptor, relative, name), so verification still spends three Python frames per folder level and makes no C-level call. (The first version used afunctools.partial, which re-entered the interpreter from C at every level.) Lean capture sets nomax_depth, so that count decides how deep a tree fits under the recursion limit: review found that a lambda's extra frame lowered the deepest single-folder chain a Lean capture accepts at Python's default limit from 328 folders to 246. Passing the callback directly keeps it at 328 on 3.13 and 3.10, as on main. The entry and child-folder checks that both verification loops reached are oneverify_recordedclosure. Docstrings that restatedTreeSelectionare shorter.lean.py(+18/-81). Since Prove Lean capture stability only for Lean inputs, and name what changed #257 the Lean selection admits only.leanentries outside excluded folders, so_indexed_source_snapshot's suffix and exclusion filters dropped nothing and_lean_generation_revisionrebuilt a snapshot equal to its input. Both are gone, withBoundProjectSources.excluded, their only input. An instrumented run of the full suite on main checked all 779 Lean index builds: the filters never dropped an entry and the two revisions always matched. The retry loop closes a failed binding in one handler instead of three.Coverage
68 named mutants of the selected-mode code, collected from #257's review rounds plus new ones for branches no earlier set covered; kill suite
test_lean_sources,test_tree_snapshot_limits,test_impact. An independent verifier re-ran all of it:The same 13 survive in every run, so none is new; they are corners #257 left untested, mutants that are equivalent or need a race inside one call, and two placeholder mutants no Lean selection reaches. Closing them means adding tests, outside this PR. The 37 filesystem-history probes from #257's review print byte-identical output on main and on this branch. Capturing hatcher-at, proximity-prize, formal-math and zeta7-irrational read-only, five exclusion variants each, gives the same revisions, generation revisions, file counts and declaration counts on main and this branch. The deepest single-folder chain a Lean capture accepts at Python's default recursion limit is 328 folders on main and on this branch, on 3.13 and 3.10; 329 fails on both with "directory tree is nested too deeply to capture". Every #257 test case has a successor here that fails on pre-#257 main (5cfa1f6) or kills the same mutants.
Options for you
Not made here, because each changes what a capture detects, tolerates or costs. Line counts are measured on
_tree_snapshot.pyby applying each cut to this branch.max_entries. 3 cases fail..leanfile and back). 11 cases fail; 2 probes change from raised to captured.Tests
test_impact,test_doctorandtest_lean_root, merged with main (dff9cfc): 398 passed, 10 skipped, 1 xfailed on Python 3.13 and 3.10. Main gives 399: two cases removed as duplicates, one test added.test_library_generatecome from the runner not installing the project and occur on main too.ruff check autoform_cli servers tests: clean.Overlap
Merges cleanly with main, which now includes #294, and with every other open PR that merges with main. Draft #90 already conflicts with main (
autoform_cli/graph.py,autoform_cli/lean.py,tests/test_graph.py); this PR addsautoform_cli/lean.pyhunks there, because #90 passesself.excludedto_indexed_source_snapshot, which this PR removes, so drop that argument when resolving. The test counts above were measured against main dff9cfc; CI on the current head is the up-to-date run.Review fixes
Two commits, from the review at #307 (comment), pushed at the user's request:
_recheck_scanned_directorytakes a three-argumentverify_kept, and verification passesverify_recordeddirectly. The partial kept three Python frames per level, but each level spent CPython's C recursion budget, so with a raised recursion limit a capture failed where main's succeeded.unkept._at("relist F")fires only on the scan's relist (the call that passes noverify_kept). Each table row asserts that its hook ran, so the two captured-once rows using it now fail if the end-of-scan relist is removed. Before, the hook fired at verification instead and the rows passed.test_selected_capture_releases_its_state_without_cyclic_gcpins selected-mode release without the cyclic collector.On a scratch run of these two commits, the unit suites passed (3,000 on 3.13, 3,001 on 3.10, 300 on Windows). Three mutants were each killed: skipping the end-of-scan relist, verification skipping kept names, and keeping the verification cycle.