docs-jenner: skip notebook re-execution (~40 min → ~1 min), add opt-in -execute variant

[jonathanhhb]

Motivation

`make docs-jenner` spends ~30 of its ~40-minute wall clock re-executing every notebook (via `docs/execute_notebooks.py`) — for a build whose only purpose is to produce `combined_mkdocs.md` for the jenner-generic-mcp RAG corpus. Every notebook is already committed with its outputs (as required for the mkdocs-jupyter render on the published docs site), so re-executing them to re-embed the same outputs in the corpus is wasted CI time.

Change

• `docs-jenner` now uses committed notebook outputs (reads notebooks directly from `docs/` instead of `dist/executed_nbs/`). No `docs/execute_notebooks.py` invocation. Wall clock ~40 min → ~1 min.
• New `docs-jenner-execute` target preserves the historical full pipeline (execute + error-gate + build + concat) for when a caller wants notebook validation as part of the doc build — typical for release-time builds where we want to catch API drift between notebooks and the current package.
• `Build Combined Doc` workflow gains an `execute_notebooks` boolean input (default false). When false → fast `docs-jenner`. When true → slow `docs-jenner-execute` with the existing `allow_notebook_errors` and `nb_timeout` inputs applied.
• Help text and `.PHONY` line updated.

What we lose (and how to keep it)

The historical `docs-jenner` implicitly validated that every notebook still runs against the current API — `check_executed_nbs.py` would error the build if any cell threw. That's useful and it's now opt-in via `docs-jenner-execute` rather than baked into every build. Follow-ups worth considering separately:

• A dedicated periodic notebook-validation workflow (weekly cron or nightly) that runs `docs-check-nbs` and posts an issue on failure, so API-drift catches don't depend on someone remembering to dispatch the -execute variant.
• Add an output-completeness check to `check_executed_nbs.py` (each code cell has at least one output) so intentionally-stripped notebooks are flagged before they land in the corpus.

Verified

• `make docs-jenner` on a clean checkout produces a valid `dist/combined_mkdocs.md` in ~1 min (mkdocs build + concat only).
• `make docs-jenner-execute` still runs the full historical pipeline.
• Downstream jenner-generic-mcp alpha suite scores 18/21 with 8 a1 against the committed-outputs corpus, matching or slightly beating the fresh-exec baseline (19/21 with 5-7 a1) on the same stripped-server config.

[Copilot]

Pull request overview

This PR speeds up the docs-jenner documentation corpus build by skipping notebook re-execution and instead consuming the outputs already committed in docs/**/*.ipynb, while providing an opt-in docs-jenner-execute path (and workflow input) to restore the historical “execute + gate + build + concat” validation when desired.

Changes:

• Update docs-jenner to skip docs/execute_notebooks.py and concatenate directly from docs/ notebook outputs.
• Add docs-jenner-execute to preserve the full historical pipeline for validation/release-time builds.
• Add an execute_notebooks boolean input to the “Build Combined Doc” workflow to select fast vs. full pipeline.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
Makefile	Splits fast vs. full combined-doc pipelines and updates help/phony targets accordingly.
.github/workflows/build-combined-doc.yml	Adds a dispatch input to select fast build vs. notebook re-execution build path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jonathanhhb]

Closing as obsolete

The CI-side wall-clock problem this PR was solving (~40 min doc builds re-executing every notebook) has been addressed differently — and, I now think, more cleanly — by #241:

• Notebook execution moved into its own workflow (execute-notebooks.yml), source-hash-cached, publishes an executed_nbs artifact (verified working end-to-end at run 29056479852).
• build-combined-doc.yml was completely rewritten to consume that artifact instead of re-executing, wall clock down to ~3-5 min on cache hit / artifact download.
• New make docs-jenner-artifact Makefile target for the CI-side "assume $(EXEC_DIR) is pre-populated" path (already on main via ci: split notebook execution into its own workflow with cached artifact (closes #186) #241).

The overlap with #241:

• docs-jenner: skip notebook re-execution (~40 min → ~1 min), add opt-in -execute variant #237's build-combined-doc.yml changes are now obsolete — the file has been rewritten and the execute_notebooks: boolean input no longer exists (it's now the choice between which workflow you dispatch, not a boolean input on one). This is what's driving the current CONFLICTING/DIRTY mergestate.
• docs-jenner: skip notebook re-execution (~40 min → ~1 min), add opt-in -execute variant #237's docs-jenner Makefile change (skip re-execution locally, read committed outputs) has a subtle footgun under the newly-adopted "committed outputs are decorative, source is authoritative" policy from ci: split notebook execution into its own workflow with cached artifact (closes #186) #241: a local make docs-jenner after a notebook edit would render stale committed figures without warning. CI has the artifact-freshness guarantee; local dev doesn't. Someone could preview a PR doc that looks correct but has old plots.

If a local-dev fast path is wanted later, a follow-up PR that adds a distinctly-named target (e.g. docs-jenner-fast) with an explicit "may render stale figures" doc comment would be the right shape.

Branch will be deleted — the diff is captured in the commit history if we ever want to salvage.

Related PRs:

• ci: split notebook execution into its own workflow with cached artifact (closes #186) #241 — CI plumbing (merged)
• hotfix: manifest.json write — pass GHA values via env vars (avoids lowercase-boolean NameError) #244 — manifest-write hotfix (merged)
• docs: surface Execute Notebooks artifact via badge + folder README #243 — README pointers surfacing the artifact (merged)
• Issue Update GHA for doc PRs and merges to execute notebooks #186 — closed by ci: split notebook execution into its own workflow with cached artifact (closes #186) #241