Make check reject Sources links to hidden paths

[Deicyde]

Closes #304.

Problem

Render omits every blueprint path with a part starting with ., but autoform check tested a ## Sources target against the real vault, so a link to a committed hidden file such as blueprint/.drafts/notes.md passed. Outside sources/, render then rewrote the link to a page it never writes, and mkdocs build --strict in the Pages workflow failed on the chapter page that absorbed the article. autoform audit reads the real vault too, so nothing named the link before MkDocs did. The Roadmap and Formalize skills already say never to link a hidden implementation note from an article.

Change

• local_target_issue reports a target as hidden-<label>-target when the path the link spells, once its .. steps are applied, has a part below the blueprint starting with . (render's _is_hidden test). Render resolves links in a copy of the vault that has neither hidden paths nor symbolic links, so that is the path it publishes: ../.x/../sources/paper.md passes, and ../.link/paper.md, with .link a symlink to sources, fails although it reaches a published file. The issue suggested judging the resolved path; the two differ only through a symbolic link. A hidden folder above the blueprint does not count, and a hidden page may link to a hidden path, since render publishes neither.
• check rejects hidden-source-target, naming the article and the link: error: result: source target names a hidden path, which render omits: '../.drafts/notes.md'.
• Because the rule lives in local_target_issue, audit reports every Sources link that check rejects, as in Make check reject Sources links that name no file in the blueprint #287, and its coverage scan reports hidden-coverage-target. Render already refused such a link in DECOMPOSED evidence, since its copy of the vault holds no hidden file; its message now says the target is hidden, and the audit's coverage contract check, which reads the real vault, now refuses the link too.
• Every hidden target is rejected, including one under sources/, as the issue recommends. Render is unchanged. Whether a hidden target under sources/ should fail is still open; see below.
• The Validation section of the CLI reference describes the new rule.

A hidden note under sources/

On main, an article that cites a committed hidden note such as ../sources/.private/paper.md passes check, and in CI render links the note to the repository at the verified commit, where the file exists, so the published link works. With this change, check exits 1 on that link. Both generated workflows run check before render or the Lean build, so such a project stops publishing and every verify run fails until the note moves out of the hidden folder. That is the kind of break #325 undid: it made check warn and keep its exit code for a Sources link that work and search leave out, because Autoform's tools must not break a codebase.

The maintainer has not decided yet whether check should fail on a hidden target under sources/. The review on #326 asks the same question from render's side, so the two are to be decided together:

• (a) Fail, as this PR does. Render omits hidden files, and a repository link into the blueprint needs the note's bytes in the linked commit, which render cannot check for a note its capture skips. Exempting sources/ would make whether the link works depend on repository coordinates that check does not see; without them, as in a local render of uncommitted changes, the strict build already fails on such a link. After Rewrite angle-bracket and query links whole, and link only captured notes #326, which links a sources/ target to the repository only when render's capture holds it, the link fails the strict build in CI too, so under (a) this PR lands after Rewrite angle-bracket and query links whole, and link only captured notes #326.
• (b) Fail only outside sources/, and print a warning: for a hidden target under sources/ through the channel Let work and search omit what they can't represent, and have check warn #325 added. A link that deploys today keeps deploying, as Let work and search omit what they can't represent, and have check warn #325 asks, but this is consistent only if Rewrite angle-bracket and query links whole, and link only captured notes #326 also keeps linking hidden notes to the repository.

Tests

• test_check_rejects_sources_links_that_name_no_file_in_the_blueprint now also checks that ../.drafts/notes.md and ../sources/.private/paper.md fail although both files exist, and that ../.x/../sources/paper.md passes.
• test_audit_validates_local_source_links_without_network_access now also expects hidden-source-target for ../../.drafts/notes.md.
• test_local_targets_are_resolved_against_the_boundary covers the coverage label and a hidden page's link to its own heading, which passes. The new test_a_link_through_a_hidden_symlink_is_hidden covers a link through a hidden directory symlink, in a vault that itself sits inside a hidden folder.

The test suite was not run locally; CI runs it. Called directly, outside pytest, these four tests pass on this commit, and each fails with main's local_target_issue at d1ad2fb. ruff check autoform_cli servers tests passes.

Landing

Rebased onto main at d1ad2fb and merges cleanly. #337, merged into main after the last rebase, conflicted in one hunk of markdown.py: the rebase keeps this PR's hidden-target check right after the escape check and follows it with #337's if not os.path.isfile(candidate):, without the try.

Checked with git merge-tree --write-tree, merging each of these PR heads into this branch and into main: #213, #214, #299, #320, #322, #324, #326, #327, #328, #329, #333, #334, #339 and #340 merge cleanly into both; #90, #195, #215, #216, #217, #219 and #284 conflict with main, with the same conflict blocks with and without this PR. #320 makes the graph read Sources links with a code-only label or a reference, which then reach this rule through node.sources with no further change.

Under (a) this PR lands after #326; under (b) no order is needed.

[Deicyde]

Review at 4e55814

Verdict: Request changes. check now rejects a ## Sources link whose target below the blueprint is hidden, and audit reports it too through the shared local_target_issue, so a link to a committed .drafts/notes.md fails check, naming the article, instead of failing mkdocs build --strict on the chapter page. Outside sources/ this matches #287, since render publishes such a link to a page it never writes. Under sources/ it fails a project whose link works in CI today, the pattern #325 reverses, and the body neither weighs that against #325 nor lists #325's conflicts; the README also misdescribes one tested case.

Fixes before merge

1. Weigh Let work and search omit what they can't represent, and have check warn #325 in the body. On main, a committed hidden note under sources/ that an article cites passes check, and in CI, render links it to the repository at the verified commit (autoform_cli/render.py:2063-2064), where the file exists, so the published link works (confirmed by trace). With this PR, check exits 1 on that link (tests/test_graph.py:564-569). Both generated workflows run check before render or the Lean build, so the project stops publishing and every verify run fails. Record the outcome of the decision below under Change. In the body's Landing section, replace "every open PR head" with the PRs actually checked and add Let work and search omit what they can't represent, and have check warn #325, as in Landing below.
2. The README says "as written" (autoform_cli/README.md:972-974). The check test expects ../.x/../sources/paper.md to pass (tests/test_graph.py:550), although its text contains .x, and check: a Sources link to a hidden file passes, then fails mkdocs build --strict #304 asked for the resolved path to be judged, not the link text. Suggested wording: "So does a target whose path below the blueprint has a part starting with . once its .. steps are applied, before or after symbolic links resolve, because render omits hidden files and directories." If the resolved-path half goes (first Optional item), drop "before or after symbolic links resolve".

Optional

• autoform_cli/markdown.py:646-654: the resolved path adds a rejection only for a link through a symlink whose own path is visible. Render refuses such a symlink inside the blueprint (autoform_cli/render.py:678) and keeps only the text of a link through one outside it. Judging the spelled path alone would remove the second candidate, the "or resolves to" wording, and the visible-symlink case at tests/test_markdown.py:424 and :434, without letting a broken link through.
• check loads articles in hidden roadmap folders, which render omits (autoform_cli/graph.py:288). Such an article citing a sibling hidden file now fails check although render publishes neither file (confirmed by trace). This is rare; a follow-up issue on that loading fits better than a special case here.
• autoform_cli/markdown.py:648 copies render's _is_hidden (autoform_cli/render.py:708). Render already imports from markdown.py, so the predicate can live there once. The summary line at autoform_cli/markdown.py:613 still describes only links that do not resolve.
• tests/test_audit.py:518 repeats the check test's three links through the same function. Only its vault inside a hidden folder is new, and that case fits in the tests/test_markdown.py test.

Landing

• Let work and search omit what they can't represent, and have check warn #325 conflicts in four hunks: the README's Validation paragraph, the _source_link_issues docstring, the rejected table (Let work and search omit what they can't represent, and have check warn #325 deletes unsupported-source-link, and this PR adds its entry right after that line) and the Sources-link test. Whichever lands second keeps Let work and search omit what they can't represent, and have check warn #325's three codes plus hidden-source-target, Let work and search omit what they can't represent, and have check warn #325's docstring and README text plus this PR's sentence on hidden targets, and Let work and search omit what they can't represent, and have check warn #325's mailto line and warning assertion plus the Same paper line.
• Leave malformed link targets as written in render and the runtime #327 no longer conflicts: it was narrowed to render and the runtime, and Report malformed Sources and dependency links instead of crashing #337 carries the shared link guards, so the body's Leave malformed link targets as written in render and the runtime #327 resolution can go. Read code-only labels, reference links and nested brackets as links #320, Stop worker groups that Lean's watchdog leaves running at LSP close #322, Rewrite angle-bracket and query links whole, and link only captured notes #326, Leave malformed link targets as written in render and the runtime #327, Make check reject a container article that sets a declaration #328 and Say how unused-hypotheses differs from Batteries' unusedArguments linter #329 merge cleanly. The order follows the decision: under (a), after Rewrite angle-bracket and query links whole, and link only captured notes #326; under (b), after Let work and search omit what they can't represent, and have check warn #325.

Decision for the maintainer

• Should check fail on a hidden Sources target under sources/? (a) Yes, as here. Render omits hidden files (autoform_cli/README.md:1852-1853), and a repository link into the blueprint needs the note's bytes in the linked commit (:1873-1874), which render cannot check for a note its capture skips. After Rewrite angle-bracket and query links whole, and link only captured notes #326, the same link fails the strict build in CI anyway. (b) Fail only outside sources/, and print a warning: for a hidden sources/ target through Let work and search omit what they can't represent, and have check warn #325's channel. That is consistent only if Rewrite angle-bracket and query links whole, and link only captured notes #326 also keeps linking hidden notes. The Rewrite angle-bracket and query links whole, and link only captured notes #326 review asks the same question from render's side, so decide the two together. Recommendation: if a link that deploys today must keep deploying, as Let work and search omit what they can't represent, and have check warn #325 asks, take (b) here and option (b) in Rewrite angle-bracket and query links whole, and link only captured notes #326's review. Otherwise (a), landed after Rewrite angle-bracket and query links whole, and link only captured notes #326, with the body naming the case that passes today and fails afterwards (a committed hidden note under sources/ cited from Sources) and its fix, moving the note out of the hidden folder.

This PR came from the same PR swarm that reviewed it, so this is the author's review, not an independent one. It was a static review: no tests were run locally. CI at this head: all 7 checks pass, including the real-Lean jobs.

Posted by PR swarm: PR Swarm Lead

[Deicyde]

Review changes addressed at e485c30

• Rebased onto main at feec945, which includes Let work and search omit what they can't represent, and have check warn #325. The four hunks that conflicted keep Let work and search omit what they can't represent, and have check warn #325's rejected table, docstring, README text and test lines, plus this PR's hidden-target entry, sentences and Same paper link.
• README, Validation: a hidden target is now one whose path below the blueprint has a part starting with . once its .. steps are applied, before or after symbolic links resolve. This replaces "as written or as resolved", which the tested ../.x/../sources/paper.md contradicted.
• Body: a new subsection, "A hidden note under sources/", names the case that passes on main and fails with this PR (a committed hidden note under sources/ cited from Sources), its fix, and options (a) and (b).
• Body, Landing: it now lists the PRs checked with git merge-tree instead of "every open PR head", records Let work and search omit what they can't represent, and have check warn #325's conflicts in the past tense, and drops the Leave malformed link targets as written in render and the runtime #327 resolution, since Leave malformed link targets as written in render and the runtime #327 no longer touches these files.
• local_target_issue and the tests this PR adds are unchanged.

Still open for you: whether check should reject a hidden note under sources/ or let it keep working with a warning. #326's review asks the same question from render's side, so the two are best decided together.

CI is running on this head.

Posted by PR swarm: PR Swarm Lead

[Deicyde]

Follow-up review at 46d90b3

Verdict: Ready to merge. This pass rechecked the last round's fixes and the whole diff against main. check and audit now report a link as hidden only when a page render publishes names a hidden path once the link's .. steps are applied, which is the path render writes, so links on hidden pages, a vault inside a hidden folder, and ../.x/../sources/paper.md all pass as on main. The one blocker, a conflict with #337 after main merged it, is resolved: the branch is rebased onto main at d1ad2fb.

Changed in this pass

• Rebased onto main at d1ad2fb, after Report malformed Sources and dependency links instead of crashing #337. In local_target_issue the hidden-target block now sits right after the escape check and is followed by Report malformed Sources and dependency links instead of crashing #337's if not os.path.isfile(candidate):, which replaces this branch's try/is_file block.
• local_target_issue judges only the path the link spells (autoform_cli/markdown.py:653-658). The resolved-path half only added two rejections: a visible symlink to a hidden file, which render's capture refuses anyway, and a link out through a symlink outside the blueprint, which render publishes as plain text.
• A link on a hidden page is not reported as hidden: a hidden page's link to its own heading failed check, though render publishes neither (tests/test_markdown.py:408-409).
• The message now reads "names a hidden path, which render omits", and the local_target_issue summary mentions hidden paths.
• The symlink test now puts the vault inside a hidden folder (tests/test_markdown.py:414), and the separate audit test became one more link in test_audit_validates_local_source_links_without_network_access (tests/test_audit.py:464-491).
• README (autoform_cli/README.md:973-976): the new sentence no longer reflows the rest of the paragraph, and it limits the exemption to this rule ("unless the article citing it is hidden too").
• The body and commit message now describe the spelled-path rule and the current merge results.

Decision for the maintainer

• Should a hidden note under sources/ fail, as here (a), or warn (b)? Decide it together with Rewrite angle-bracket and query links whole, and link only captured notes #326's review. The recommendation stands: (b) if a link that deploys today must keep deploying, as Let work and search omit what they can't represent, and have check warn #325 asks; otherwise (a), landed after Rewrite angle-bracket and query links whole, and link only captured notes #326.

This PR came from the same PR swarm that reviewed it, so this is the author's review, not an independent one. No tests ran under pytest locally. After the rebase, the changed check, Markdown and audit tests and #337's two new audit tests were called directly on this head and pass, and ruff check passes. CI at this head: all 7 checks pass, including the real-Lean jobs.

Posted by PR swarm: PR Swarm Lead