Repository navigation
Skip rendered sites in every Lean capture - #336
Merged
Merged
Conversation
render copies every non-Markdown file under the blueprint into its site, so a .lean file kept in the blueprint lands in the site, and mkdocs then copies the site, publication.json included, into site/. Only render's own Lean capture treated a folder holding a complete publication.json as generated output: it passed the marker through the opaque_markers parameter, while check, audit, doctor, work, search, skeleton, runtime, impact and unused-hypotheses used the default selection, whose markers were only .git and the skeleton manifest. Those commands indexed the copies, so they saw duplicate declarations, kept resolving names the source had since renamed, and computed a different Lean revision from render's for the same tree. The comment on render's marker claimed every Lean walk skipped sites. Move the publication manifest's name, schema, byte cap and completeness check into lean.py and add the marker to the default selection, next to the skeleton manifest, so every capture skips a site and its mkdocs copy. render now imports these names from lean.py and still re-exports them. render was the only caller of opaque_markers, so the parameter and its plumbing through snapshot_project_sources, open_project_sources and _lean_tree_selection are gone. render keeps its copy-root exclusion and refusal rules. The README stop list now says every command skips such a folder. render writes a complete manifest into its stage before it copies any file, so a capture during a render skips the stage too (pinned by a test that captures after each copy). mkdocs build empties site/ and copies files in name order, so site/ can hold copies before its publication.json; the README tells users to rerun a command that indexed during the build.
Contributor
Author
|
Merge readiness: one small change requested (reviewed at f48f40a) What holds (independent review plus CI, green on every job including Windows and the real-Lean jobs):
Change requested:
Body nits:
It is a draft, so it needs to be marked ready before merging. |
render writes a complete publication.json into its stage before it copies any file, so a Lean capture during the render skips the stage. Once every page exists it rewrote the manifest with Path.write_bytes, which truncates the file before writing the new bytes. A capture that read the marker in that gap found an empty file, did not recognize the folder as a site, and indexed the stage's .lean copies. The README's claim that a command indexing during a render skips the site was therefore not quite true (review comment on #336). _write_publication_manifest now writes the bytes to a hidden temporary file in the same folder, opened with O_EXCL and mode 0o666 under the umask as write_bytes would, and os.replace moves it onto publication.json, so a reader sees either the previous complete manifest or the new one. The temporary file is removed if anything fails. Its name is not a marker and not a .lean file. test_a_capture_while_render_rewrites_its_manifest_skips_the_stage captures just after an in-place write would truncate the manifest and on both sides of the replace, and asserts that only the sources are indexed. It fails on the parent, where the capture includes the stage's Proofs.lean. test_only_a_complete_publication_manifest_hides_its_folder now patches the byte cap with raising=False, so on main it fails at its assertion (stale copies captured) instead of at the patch.
Contributor
Author
|
Update: change made, now ready to merge (b007a13)
Optional: the new test catches a truncating rewrite through It is a draft, so it needs to be marked ready before merging. |
This was referenced Oct 15, 2026
Deicyde
marked this pull request as ready for review
October 15, 2026 17:19
Deicyde
added a commit
that referenced
this pull request
Oct 15, 2026
Main brings #331, #332, #335, #341, #338, #336, #323, #343 and #337. lean.py, render.py and tests/test_render.py conflicted, and four changes were needed outside the conflicts: - lean.py: snapshot_project_sources and _open_stable_project_sources lose the opaque_markers parameter #336 removed and keep this branch's names= keyword. The publication constants #336 moved here take this branch's autoform-publication/v3, and _REPLACEABLE_SCHEMAS moves with _is_complete_publication, since lean.py cannot import render.py. Every Lean capture skips a complete v1, v2 or v3 site; with main's v2 alone it would index this branch's sites and their mkdocs copies. - render.py: imports the constants and _REPLACEABLE_SCHEMAS from lean.py, drops its own _is_complete_publication, and its Lean capture passes names= and relies on the marker every capture now has. - tests/test_render.py: keeps both sides' tests. #336's audit test forwards the names= keyword audit passes, and this branch's revision test builds TreeSnapshot without root_identity and placeholders, which #341 deleted. - graph.py: _catalog_has_local_ledger takes #337's guards (a link urlsplit rejects is skipped, RuntimeError from resolve is caught, os.path.isfile replaces Path.is_file), so a catalog's Sources link can no longer crash load_graph. A new test in tests/test_graph.py covers an invalid URL, an over-long name and a symlink loop.
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
autoform rendercopies every non-Markdown file under the blueprint into itssite, so a
.leanfile kept in the blueprint lands in the site.mkdocs buildthen copies the site,
publication.jsonincluded, intosite/. Only render'sown Lean capture treated a folder holding a complete
publication.jsonasgenerated output; it passed that marker through an
opaque_markersparameterno other caller used.
check,audit,doctor,work,search,skeleton,runtime,impactandunused-hypothesesused the default selection (only.gitand the skeletonmanifest.json), so they indexed the copies: duplicatedeclarations, stale names that still resolved after the source renamed them,
and a different Lean revision from render's for the same tree. Found in the
#307 post-merge review.
This PR treats a folder holding a complete
publication.jsonas generatedoutput in every Lean capture, exactly like skeleton output.
autoform_cli/lean.py:PUBLICATION_MANIFEST,PUBLICATION_SCHEMA,PUBLICATION_MANIFEST_MAX_BYTESand_is_complete_publicationmove herefrom
render.py(render already importslean.py, not the other way round),and the marker joins
_lean_tree_selection's default markers.opaque_markersis removed fromsnapshot_project_sources,_open_stable_project_sources,open_project_sourcesand_lean_tree_selection; render was its only caller (checkedautoform_cli,servers,skills,tests).autoform_cli/render.pyimports the names fromlean.pyand still exportsthem (
dashboard.pyand tests import them fromrender). It keepsexclude_roots=(copy_root,)and its refusal rules. The comment that claimedevery Lean walk skipped sites is replaced by one at the capture call that is
now true.
_write_publication_manifestreplacespublication.jsonthrougha hidden temporary file in the same folder and
os.replaceinstead ofrewriting it in place, so a capture never reads a truncated marker. Bytes
and file mode are unchanged.
autoform_cli/README.md: the stop list no longer says "forrender"; everycommand skips a folder holding a complete
publication.jsonand stops at anunreadable one.
Behavior changes
complete
publication.json(schema v1 or v2,"complete": true, at most8 MiB), so render and the other commands index the same files and agree on
the Lean revision.
publication.jsonbelow the Leanroot, as render already did and as all commands do for
manifest.json.publication.jsonhides nothing.writes a complete
publication.jsoninto its stage before it copies anyfile, and replaces it, never truncates it, when it adds the file inventory.
Not changed here:
mkdocs buildemptiessite/and copies files inname order, so
site/can hold a copied.leanfile before itspublication.json(checked with mkdocs 1.6.1); a command that indexes duringthe build can read it, and running it again afterwards gives the settled
result. The README says so.
Tests
tests/test_lean_sources.py::test_only_a_complete_publication_manifest_hides_its_folder:one tree with render-style
site-src/, mkdocs-stylesite/, a v1 site, andincomplete, unrelated and oversized manifests, each over a stale
.leancopy; asserts exactly which sources
snapshot_project_sourcescaptures.Fails on main (stale copies under
site-src/,site/andlegacy/arecaptured).
tests/test_render.py::test_audit_and_render_skip_a_site_and_its_mkdocs_copy_alike:renders into
site-src/, copies it tosite/as mkdocs does, renames thesource declaration, renders again, then audits a
lean:that names the olddeclaration. Asserts audit's Lean revision equals render's
lean_source_revisionand that audit reportslean-target-not-found. Bothassertions fail on main.
tests/test_render.py::test_a_capture_during_a_render_skips_the_stage_being_written:captures after each
.leancopy during a render and asserts only the sourcesare indexed. Fails on main (the capture includes
.autoform-publication-<hex>/Proofs.lean).tests/test_render.py::test_a_capture_while_render_rewrites_its_manifest_skips_the_stage:captures during render's final manifest write (just after an in-place write
would truncate the file, and on both sides of the replace) and asserts only
the sources are indexed. Fails when the manifest is rewritten with
Path.write_bytes(the capture includes.autoform-publication-<hex>/Proofs.lean).test_render,test_lean_sources,test_windows_source_boundary,test_dashboard,test_audit): 499passed, 5 skipped, 1 xfailed on Python 3.13 and on 3.10.
All 48 are in
tests/test_library_generate.pywith "Autoform is notinstalled": the local venv is synced with
--no-install-project, and mainfails the same 48 in that venv.
ruff check autoform_cli servers testsis clean.Overlap with open PRs
MAIN_REF=upstream/main ~/bin/pr_conflicts.shon the merge of this branchinto upstream/main feec945 (the branch is based on e108ee0 and merges
cleanly into feec945). New overlaps, all with draft PRs that already
conflict with main on
graph.py:lean.py3 hunks,render.py3,tests/test_render.py1.lean.py2,README.md2,tests/test_lean_sources.py1.No new overlap with any other open PR (#195, #213 to #216, #320 to #335, #337, #338).