Skip to content

ci: cut the runtime of the nightly link check - #1215

Open
lundybernard wants to merge 4 commits into
napari:mainfrom
lundybernard:linkcheck-nightly-runtime
Open

lundybernard wants to merge 4 commits into
napari:mainfrom
lundybernard:linkcheck-nightly-runtime

Conversation

@lundybernard

@lundybernard lundybernard commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Much of the link phase is idle time after github.com answers with 429.
Every run skips notebook execution and the commits?author= links, and
checks GitHub code links through the REST API with the workflow token.
The report artifact gains output.json, with the status of every checked
URI.

Changes:

  • build: set NB_EXECUTION_MODE off for linkcheck-files

    The link check reads the links in the page sources, not in the
    notebook outputs. Notebook execution adds build time but does not add
    links to check.

  • perf: add GitHub commits?author= links to linkcheck_ignore

    The docs hold 325 unique commits?author= links, most of them in the
    release notes. Each check spends GitHub rate-limit budget, and none of
    them can report a broken link.

  • ci: upload output.json with the link check report

    output.txt lists only the broken, redirected, and timed-out links.
    output.json holds every checked URI and its status.

  • perf: check GitHub code links through the REST API

    github.com answers most of its 429s on blob and tree pages, and it
    ignores a token on HTML pages. With GITHUB_TOKEN set, the
    linkcheck-process-uri hook rewrites each blob or tree link to its
    contents URI on api.github.com, which accepts the token. The workflow
    sets the token. A run without it checks the github.com page, as before.
    The check drops the page anchor, and a failure reports under the API
    URI.

Refs: #1179
Assisted-by: claude-code:claude-opus-5-5 [The Engineer]

@github-actions github-actions Bot added documentation Improvements or additions to documentation maintenance CI, dependencies, and other maintenance labels Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

@lundybernard

lundybernard commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Detailed PR notes


🤖 AI text below 🤖

This is the runtime PR for #1179.
It carries options A, B1, D', and E2 from the technical report linked in #1179,
plus the output.json upload, rebuilt on main.
It does not stub the generated pages (option E1 in the report).
Instead, #1212 cuts the same build time and keeps the npe2 pages.

Merge order: any.
The branch merges cleanly with main and with the open #1212, #1204, and #1211 (git merge-tree --write-tree, exit 0 each).
The build-time saving on the generated pages comes from #1212, not from this branch.
No linkcheck_ignore pattern on this branch duplicates one on main.

Baseline, measured on 2026-10-27, before #1207 merged and before #1212.
The main row ran on napari/docs, the other three on a fork.
I'll rerun the timings on main after #1212 lands and update this table.

Run Configuration Scope Time
40424058983 main every page 51 min
40574046854 config only nightly 19m38s
40574493815 with the REST API rewrite nightly 6m54s
40580943817 with the REST API rewrite full 17m25s

The three fork runs differ from this branch.
Their nightly scope skipped every release note; this branch keeps the newest two minor series.
They used the stub step and a pip-cache step, which this branch leaves out.
Their tree sits on the main of 2026-10-26, with the #1195 and #1196 changes on top.

Verification:

  • At each commit, docs/conf.py parses and linkcheck.yml loads as YAML.
    actionlint passes on the workflow.
  • On the current release files, the nightly scope keeps release_0_10_0, release_0_9_0, release_0_9_1, and release_0_9_2,
    and skips the other 61 release notes.
  • The commits?author= pattern matches every such URL under docs/.
    No linkcheck_ignore or linkcheck_allowed_redirects entry on main matches any of them.

Reviewer notes:

  • Release-note scope: Tim's comment on Improve linkchecking action #1179 (2026-11-05) proposes skipping the release notes.
    This branch skips only the notes before the newest two minor series, Monday to Saturday, and checks every page on Sundays.
    The newest notes still get edits, so they stay in the nightly, and the Sunday run keeps the older notes checked.
    The branch keeps this rule unless a reviewer prefers the full skip.
  • Token: the workflow passes the job's github.token as GITHUB_TOKEN.
    conf.py sends it only to api.github.com, through linkcheck_request_headers.
    Without the token, the check reads the github.com pages as before.
    A broken code link reports under its API URI, without the page anchor.
  • Notebook execution: the skip joins the Makefile export line that the slim targets use, so local make linkcheck-files runs skip execution too.
  • A manual run takes a scope input: nightly (the default) or full.
  • perf: add napari pull and issue links to linkcheck_ignore #1214 adds a linkcheck_ignore pattern for the project's own pull and issue links; the two merge in either order.

Comment thread docs/conf.py
return None


def rewrite_github_code_link(uri: str) -> str | None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that we've updated the ignore/link style for the github urls, do we still want to keep this rewrite of github urls? mostly asking naively without looking too closely.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'd keep it. It covers blob and tree links, which #1214 doesn't touch.
A run checks 110 of those, or 108 once the anchors drop.
On main they cause 21 to 37 rate-limit waits a run, none with the rewrite.
In my trial runs it cuts the link check step ~8, to ~5.

@TimMonko TimMonko left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Can we get a re-timing on these now that we've cleaned up the links (and the docs CI). I'm wondering how much time we are saving now that we aren't rate limited by Github and such with all the recent PRs, so therefore whether its worth adding a bit more complexity for the nightly time save.

Comment thread docs/conf.py Outdated
Comment on lines +768 to +770
# A nightly link check skips the release notes before the newest two minor
# series. Without LINKCHECK_SCOPE=nightly, the link check reads every page.
if os.environ.get('LINKCHECK_SCOPE') == 'nightly':

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this 👍

env:
PIP_CONSTRAINT: ${{ github.workspace }}/napari/resources/constraints/constraints_py3.12_docs.txt
LINKCHECK_SCOPE: ${{ steps.scope.outputs.scope }} # on nightly, conf.py skips the old release notes
GITHUB_TOKEN: ${{ github.token }} # conf.py sends it to api.github.com only

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this here now, but not previously? I see it's used in the makefile now but I'm curious why it's only showing up in this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token only helps on GitHub's API, not on the github.com website.
So this PR checks links to code through the API, and this line gives that check the job token. Without the token, the API stops at 60 calls an hour, and a run checks about 110 code links.
The Makefile doesn't name it, it just hands the step env to sphinx-build.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation!

@lundybernard
lundybernard force-pushed the linkcheck-nightly-runtime branch from 33bdd20 to 655b184 Compare November 17, 2026 18:24
@lundybernard

Copy link
Copy Markdown
Contributor Author

re-timing on current main, after #1212 and #1214.
Every row checks every page.

Tree Runs Total job Link check step Checked URIs Rate-limit waits on code links
main, scheduled 4 24m53s to 30m23s 21m55s to 27m29s 1265 to 1266 21 to 37
this branch 2 8m04s, 8m55s 4m32s, 5m34s 937 0, 0
this branch without the REST API rewrite 2 12m49s, 9m59s 10m01s, 6m26s 939 17, 23

I dropped the release-note skip.
After #1214 and the commits?author= ignore,
the old release notes hold 72 of the 939 checked URIs.
That doesn't pay for a scope input, a Sunday rule, and a file-name cut-off.
Every run checks every page again.

I force-pushed the branch with one commit fewer.
The release-note skip commit is gone, and the other four make the same changes.
This replaces the timing table in my first comment.
I dropped scope input and the Sunday rule and updated the PR description to reflect the change.

The link check reads the links in the page sources, not in the
notebook outputs. Notebook execution adds build time but does not add
links to check.

Refs: napari#1179
Assisted-by: claude-code:claude-opus-5-5 [The Engineer]
The docs hold 325 unique commits?author= links, most of them in the
release notes. Each check spends GitHub rate-limit budget, and none of
them can report a broken link.

Assisted-by: claude-code:claude-opus-5-5 [The Engineer]
output.txt lists only the broken, redirected, and timed-out links.
output.json holds every checked URI and its status.

Assisted-by: claude-code:claude-opus-5-5 [The Engineer]
github.com answers most of its 429s on blob and tree pages, and it
ignores a token on HTML pages. With GITHUB_TOKEN set, the
linkcheck-process-uri hook rewrites each blob or tree link to its
contents URI on api.github.com, which accepts the token. The workflow
sets the token. A run without it checks the github.com page, as before.
The check drops the page anchor, and a failure reports under the API
URI.

Assisted-by: claude-code:claude-opus-5-5 [The Engineer]
@lundybernard
lundybernard force-pushed the linkcheck-nightly-runtime branch from 655b184 to 3982419 Compare November 17, 2026 18:52
@TimMonko

Copy link
Copy Markdown
Member

Thanks for the updates @lundybernard! I'm a bit confused by your table in #1215 (comment) suggesting no rate limits, but when I look at https://github.com/3M1LY-lb/napari-docs/actions/runs/43984023760/job/133707442017#step:7:5244 it seems like we get rate-limited by Github still? Mostly trying to understand and look for clarification

Would it be helpful to add the author and org checks to the rest API? We can do that here or in a follow-up, whatever you prefer. I think this is ready to go once my questions are answered here

It seems our slow part now is down to the docs build, rather than stalling during the link checks -- so thank you! I appreciate how you slimmed down the logic to not have special rules, for now. 🍻

@lundybernard

Copy link
Copy Markdown
Contributor Author

@TimMonko My table column was too narrow. It counts the waits on code links only (blob and tree), because those are the links the REST API rewrite moves.

Yes, I think moving the user and org pages to the REST API would help.
A run checks 40 profile links. In the run you linked, the last minute and a half of the link check was only github.com links that sat behind those waits.
api.github.com/users/<name> answers for users and orgs alike,
so one more rule in the same hook covers both.

I'd rather do that in a follow-up PR, so this one can merge as it is.
I'll open it once this one is in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation maintenance CI, dependencies, and other maintenance

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants