diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 4b8aa2a8fe..b4e2dfc2dd 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -18,7 +18,7 @@ Stored the app credentials in `wandb/docs`: The workflows use `actions/create-github-app-token@v3` to create short-lived installation tokens from these credentials. -Workflows that push back to a same-repo PR branch with this token (instead of the default workflow `GITHUB_TOKEN`) include **Compress Images** (`calibreapp-image-actions.yml`), **Build CSS** (`build-css.yml`), and **Knowledgebase Nav** (`knowledgebase-nav.yml`). That way downstream `pull_request` checks (for example **Validate MDX**) still run on the automation commit. +Workflows that push back to a same-repo PR branch with this token (instead of the default workflow `GITHUB_TOKEN`) include **Compress Images** (`calibreapp-image-actions.yml`), **Build CSS** (`build-css.yml`), and **Knowledgebase Nav** (`knowledgebase-nav.yml`). That way downstream `pull_request` checks (for example **Doc quality**) still run on the automation commit. ## Action pinning @@ -192,46 +192,54 @@ PYTHONPATH=scripts python3 -m uidrift.scan decide \ `--agreement` is the detector's only feedback channel and cannot be reconstructed later. A dismissal reopens by itself if docs later start covering that surface, so it suppresses a row without hiding it forever. -## Readability delta +## Doc quality -**Workflow**: `readability-delta.yml` +**Workflow**: `doc-quality.yml` -Posts an informational, **non-blocking** PR comment describing how the PR affects the readability of the English docs it changes (DOCS-2626). It reports the *delta* (before/after) for well-established formulas (Flesch-Kincaid grade, Flesch reading ease, Gunning fog, SMOG), word-weighted across the changed pages, plus an optional AI-agent-comprehension rating from a W&B Inference LLM judge. +One workflow, one runner job, one upserted PR comment for the three checkout-based doc-quality checks. It replaces the former `validate-mdx.yml`, `linkcheck-pr.yml`, and `readability-delta.yml` workflows. The job id stays `validate-mdx` so the branch-protection required check keeps its name. -### Triggers +The three checks: -- **Pull request**: `opened`, `synchronize`, `reopened` on PRs that touch `**/*.mdx` -- **Manual**: `workflow_dispatch` (writes the report to the job summary instead of a comment) +1. **MDX validation** (**blocking**): `mint validate` + `mint broken-links` via `scripts/mdx-validation/validate-mdx-mintlify.sh`, using the Mintlify CLI from npm (cached ~4 days). `mint broken-links` owns *internal* links because it resolves them against the real route table (docs.json navigation and redirects). +2. **External link check** (informational): `lychee` over the changed `.mdx` files only, external URLs only (`lychee.toml` excludes `^file://`). Runs on plain `pull_request` from the git checkout — it does not wait for the Mintlify preview deployment (the old `deployment_status` trigger was vestigial scheduling and carried the DOCS-3041 wrong-PR-resolution bug). +3. **Readability delta** (informational, DOCS-2626): reports how the PR affects the readability of the English docs it changes — the *delta* (before/after) for well-established formulas (Flesch-Kincaid grade, Flesch reading ease, Gunning fog, SMOG), word-weighted across the changed pages, plus an optional AI-agent-comprehension rating from a W&B Inference LLM judge. Localized content under `ja/`, `ko/`, and `fr/` is skipped. -### What it does +Only MDX validation can fail the job; the other two never fail a PR. All three results land in a single PR comment identified by the `` marker (status table plus collapsed details). A migration step deletes leftover comments from the replaced workflows (`` and `` markers) on open PRs. -1. Diffs the PR base and head, scoring each changed English `.mdx` file (localized content under `ja/`, `ko/`, and `fr/` is skipped). -2. Extracts narrative prose and scores it with `textstat` via the analyzer in the `coreweave/docs-skills` submodule (`.claude/scripts/_readability.py`). -3. Optionally runs the AI agent comprehension judge (W&B Inference) when `WANDB_API_KEY` is set. -4. Upserts a single PR comment identified by the `` marker. +### Triggers -The check **never fails** a PR. If scoring is unavailable it posts a brief notice and exits successfully. +- **Pull request**: the default activity types (`opened`, `synchronize`, `reopened`), deliberately with **no `paths` filter** — a paths filter would leave the required check pending on PRs it skips. A scope-gate `git diff` inside the job fast-exits instead when a PR touches no Mintlify-relevant files (`.mdx`, `.json`, `.yaml`/`.yml`, images, or the validation script). +- **Push**: to `main` (full validation) +- **Manual**: `workflow_dispatch` (writes the report to the job summary instead of a comment). Dispatch runs the full MDX validation only: the link and readability checks diff against `main`, so on `main` itself the change list is empty and they skip. The old `linkcheck-pr.yml` dispatch fell back to a full-tree external-link sweep in that case; that behavior was **not** carried over — the closest substitute is `linkcheck-prod.yml`, which checks the live site's sitemap (monthly cron, also manually dispatchable). ### Configuration -- **Python**: 3.11 +- **Node**: 20 (Mintlify CLI), **Python**: 3.11 (readability) - **Permissions**: `contents: read`, `pull-requests: write` -- **Report glue**: `scripts/readability/pr_report.py` -- **Scoring logic**: `.claude/scripts/_readability.py` and `_docs_eval_lib.py` (submodule) +- **Validation script**: `scripts/mdx-validation/validate-mdx-mintlify.sh` +- **Link-check config**: `lychee.toml` (repo root) +- **Readability report glue**: `scripts/readability/pr_report.py` +- **Readability scoring logic**: `.claude/scripts/_readability.py` and `_docs_eval_lib.py` (submodule) ### Authentication -- The main checkout uses the default `GITHUB_TOKEN`. -- The private, cross-org `coreweave/docs-skills` submodule is initialized in a separate step with the `DOCENGINE_TOKEN` secret (the same `x-access-token` credential used for the `gitsubmodule` ecosystem in `.github/dependabot.yml`; it rotates ~every 30 days and needs no `wandb/docs` scope). +- The main checkout, the lychee rate-limit token, and PR comments use the default `GITHUB_TOKEN`. +- The private, cross-org `coreweave/docs-skills` submodule is initialized in a separate step with the `DOCENGINE_TOKEN` secret (the same `x-access-token` credential used for the `gitsubmodule` ecosystem in `.github/dependabot.yml`; it rotates ~every 30 days and needs no `wandb/docs` scope). Readability only. - The AI agent comprehension judge calls W&B Inference with the `WANDB_DOCS_INFERENCE_API_KEY` secret (a W&B API key whose entity has Inference credits), passed to the scorer as `WANDB_API_KEY`. When that secret is absent, the deterministic `textstat` delta still runs. ### Forks -Fork PRs have no access to repo secrets, so the first step detects a fork, posts an Actions notice, and makes the whole job a no-op (still reporting success). Forks are uncommon in `wandb/docs` and coreweave repos cannot use forks at all. +Fork PRs have no access to repo secrets and get a read-only `GITHUB_TOKEN`. MDX validation and the external link check need no secrets, so they still run (and validation still blocks). The readability delta skips (it needs the private submodule and the Inference key), and the report goes to the workflow step summary instead of a PR comment. Forks are uncommon in `wandb/docs` and coreweave repos cannot use forks at all. + +### Related workflows (not consolidated) + +- `mintlify-deployment-preview.yml` needs the deployment's `environment_url`, so it stays on `deployment_status`. +- `linkcheck-prod.yml` checks the live site on a monthly cron. +- The PinGuard pair and the commit-pushing bots (Compress Images, Build CSS, Knowledgebase Nav) use the `wandb-docs-pr-writer` App token — a different trust category that should not share a job with fork-facing validation. ### Related Files -- **Report glue**: `scripts/readability/pr_report.py` -- **Dependencies**: `scripts/readability/requirements.txt` -- **Tests**: `scripts/readability/tests/` -- **Documentation**: `scripts/readability/README.md` +- **Readability report glue**: `scripts/readability/pr_report.py` +- **Readability dependencies**: `scripts/readability/requirements.txt` +- **Readability tests**: `scripts/readability/tests/` +- **Readability documentation**: `scripts/readability/README.md` diff --git a/.github/workflows/build-css.yml b/.github/workflows/build-css.yml index c710e01b3d..e991ca6d13 100644 --- a/.github/workflows/build-css.yml +++ b/.github/workflows/build-css.yml @@ -34,7 +34,7 @@ jobs: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 with: # App token rather than the default GITHUB_TOKEN so the rebuild - # commit triggers downstream workflows (for example Validate MDX). + # commit triggers downstream workflows (for example Doc quality). # Fork PRs are already excluded by the job-level `if` above. token: ${{ steps.app-token.outputs.token }} ref: ${{ github.head_ref }} diff --git a/.github/workflows/calibreapp-image-actions.yml b/.github/workflows/calibreapp-image-actions.yml index 05db79df12..18f530d4f2 100644 --- a/.github/workflows/calibreapp-image-actions.yml +++ b/.github/workflows/calibreapp-image-actions.yml @@ -44,6 +44,6 @@ jobs: avifQuality: '75' minPctChange: '10' # GitHub App token with contents read-write. Unlike the default workflow `GITHUB_TOKEN`, - # commits pushed with this token trigger other workflows (for example Validate MDX). + # commits pushed with this token trigger other workflows (for example Doc quality). # Fork PRs are already excluded by the job `if` above. GITHUB_TOKEN: ${{ steps.app-token.outputs.token }} diff --git a/.github/workflows/doc-quality.yml b/.github/workflows/doc-quality.yml new file mode 100644 index 0000000000..c459473f5c --- /dev/null +++ b/.github/workflows/doc-quality.yml @@ -0,0 +1,671 @@ +# ------------------------------------------------------------------ +# Doc quality (consolidated PR checks) +# ------------------------------------------------------------------ +# +# One workflow, one runner-job, one PR comment for the three checkout-based +# doc-quality checks that used to live in three workflows: +# +# - MDX validation (BLOCKING) — `mint validate` + `mint broken-links` via +# scripts/mdx-validation/validate-mdx-mintlify.sh. Replaces validate-mdx.yml. +# The job id stays `validate-mdx` so the branch-protection required check +# name never changes. +# - External link check (informational) — lychee over the changed .mdx files, +# external URLs only (lychee.toml excludes ^file://; `mint broken-links` +# owns internal links). Replaces linkcheck-pr.yml. This version runs on +# plain `pull_request` instead of `deployment_status`: since PR #3037 the +# check reads files from the git checkout, so waiting for the Mintlify +# preview deployment was vestigial scheduling (and the source of the +# DOCS-3041 wrong-PR-resolution bug, whose code is deleted with it). +# - Readability delta (informational) — scripts/readability/pr_report.py with +# the coreweave/docs-skills analyzer. Replaces readability-delta.yml. +# +# All three results land in ONE upserted PR comment keyed by +# . Only MDX validation can fail the job. +# +# Deliberately NOT consolidated here: mintlify-deployment-preview.yml (needs +# the deployment's environment_url), linkcheck-prod.yml (cron against the live +# site), the PinGuard twin pair, and knowledgebase-nav.yml (pushes commits with +# the wandb-docs-pr-writer App token — a different trust category). +# +# Triggers mirror the old validate-mdx.yml exactly: `pull_request` with no +# paths filter (required-check safety — a paths filter would leave the required +# check pending on PRs it skips), push to main, and workflow_dispatch. +# +# Auth +# ---- +# - Main checkout and PR comments use the default GITHUB_TOKEN. +# - The coreweave/docs-skills submodule is private and cross-org, so a separate +# step initializes it with DOCENGINE_TOKEN (readability only; non-fatal). +# - The AI agent comprehension judge calls W&B Inference with the +# WANDB_DOCS_INFERENCE_API_KEY secret, passed to the scorer as WANDB_API_KEY. +# When that secret is absent the deterministic textstat delta still runs. +# +# Forks +# ----- +# Fork PRs have no repo secrets and a read-only GITHUB_TOKEN. MDX validation +# and the lychee link check need no secrets and still run (and validation still +# blocks). Readability skips (private submodule + key), and the results go to +# the workflow step summary instead of a PR comment. +# ------------------------------------------------------------------ + +name: Doc quality + +on: + pull_request: + push: + branches: + - main + workflow_dispatch: + +permissions: + contents: read + pull-requests: write + +concurrency: + # Key on the PR number, not the branch name: head_ref is the bare branch + # name, which is not unique per PR (two forks both named patch-1 collide, as + # does one branch with PRs against two bases), and a collision plus + # cancel-in-progress would cancel another PR's required check. + group: doc-quality-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + # Keep this job id: it is the check name required by branch protection. + validate-mdx: + runs-on: ubuntu-latest + # Headroom over the sum of the per-step timeouts below (lychee 10 + + # readability 10) plus checkout/mint install/validation, so two slow + # advisory steps cannot cancel the required job after validation passed. + timeout-minutes: 35 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + with: + # Full history to diff merge-base..head. + fetch-depth: 0 + submodules: false + # Do not leave the (write-capable, on same-repo PRs) token in + # .git/config while unpinned mint/lychee code runs. Nothing here + # needs it: the diffs are local, the submodule step brings its own + # DOCENGINE_TOKEN, and the gh steps get the token via env. + persist-credentials: false + + # ------------------------------------------------------------ + # 1. Scope gate: one git diff decides what runs. + # ------------------------------------------------------------ + # Pushes made with the default workflow GITHUB_TOKEN do not trigger other + # pull_request workflows. Workflows that push with the wandb-docs-pr-writer + # App token (Compress Images, Build CSS, Knowledgebase Nav) avoid that. + # Otherwise branch protection may need a human push or workflow_dispatch. + - name: Determine scope of checks + id: scope + env: + EVENT_NAME: ${{ github.event_name }} + PR_BASE: ${{ github.event.pull_request.base.sha }} + PR_HEAD: ${{ github.event.pull_request.head.sha }} + PR_IS_FORK: ${{ github.event.pull_request.head.repo.fork }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + run: | + set -euo pipefail + OUT="lychee-pr-inputs.txt" + : > "$OUT" + + if [ "$EVENT_NAME" = "pull_request" ]; then + # head.repo.fork renders as empty when head.repo is null (the fork + # was deleted), so treat anything that is not exactly "false" as a + # fork rather than letting the mutating steps 403 later. + if [ "$PR_IS_FORK" != "false" ]; then + echo "is_fork=true" >> "$GITHUB_OUTPUT" + else + echo "is_fork=false" >> "$GITHUB_OUTPUT" + fi + # The event's base.sha is the base branch TIP; diff from the + # merge-base so a PR branch that is behind main is not blamed for + # unrelated mainline changes. + MERGE_BASE=$(git merge-base "$PR_BASE" "$PR_HEAD" || echo "$PR_BASE") + HEAD_SHA="$PR_HEAD" + + # Pathspecs do not support regex alternation like *.(mdx|json); filter + # with grep instead. Includes .json for docs.json / mint.json / OpenAPI + # specs that affect generated pages. Disable rename detection so + # deleting or renaming a relevant file still triggers validation. + # Keep this regex byte-identical to MINTLIFY_RELEVANT_REGEX in + # scripts/mdx-validation/validate-mdx-mintlify.sh: if they diverge, + # this gate can run the job while the script validates nothing, or + # skip files the script considers relevant. + changed_files=$(git diff --name-only --no-renames "$MERGE_BASE" "$HEAD_SHA") + if grep -Eq '(\.(mdx|json|ya?ml|png|jpe?g|webp)$|^scripts/mdx-validation/validate-mdx-mintlify\.sh$)' <<< "$changed_files"; then + echo "mintlify_relevant=true" >> "$GITHUB_OUTPUT" + echo "Found changed Mintlify-relevant files or validation code; will run doc-quality checks." + else + echo "mintlify_relevant=false" >> "$GITHUB_OUTPUT" + echo "No Mintlify-relevant files in this PR diff; skipping all doc-quality checks." + fi + else + # push to main / workflow_dispatch: always run full Mintlify + # validation (parity with the old validate-mdx.yml). Informational + # checks diff against the default branch, which on a push to main + # yields an empty list, so they skip. + echo "is_fork=false" >> "$GITHUB_OUTPUT" + echo "mintlify_relevant=true" >> "$GITHUB_OUTPUT" + MERGE_BASE=$(git merge-base "origin/${DEFAULT_BRANCH}" HEAD || echo "") + HEAD_SHA=$(git rev-parse HEAD) + echo "Runs full validation for push/workflow_dispatch (not a PR diff)." + fi + + echo "merge_base=$MERGE_BASE" >> "$GITHUB_OUTPUT" + echo "head_sha=$HEAD_SHA" >> "$GITHUB_OUTPUT" + + # One --name-status diff feeds both .mdx outputs: + # - non-deleted .mdx files → lychee/readability input list, written + # to a file (not step outputs/argv) to dodge ARG_MAX on large PRs + # (same approach as the old linkcheck-pr.yml); + # - all changed .mdx INCLUDING deletions → gates readability, which + # renders deleted pages as "page removed" (a deletion changes the + # docset's readability even though lychee has nothing to lint). + # --no-renames means every line is "\t" (no two-path + # rename lines), so filtering deletions is a ^D check. + MDX_STATUS="" + if [ -n "$MERGE_BASE" ] && [ "$MERGE_BASE" != "$HEAD_SHA" ]; then + MDX_STATUS=$(git diff --name-status --no-renames "$MERGE_BASE" "$HEAD_SHA" \ + | { grep -E '\.mdx$' || true; }) + fi + MDX_ANY=0 + if [ -n "$MDX_STATUS" ]; then + printf '%s\n' "$MDX_STATUS" | { grep -v '^D' || true; } | cut -f2- > "$OUT" + MDX_ANY=$(printf '%s\n' "$MDX_STATUS" | wc -l | tr -d ' ') + fi + MDX_COUNT=$(wc -l < "$OUT" | tr -d ' ') + echo "mdx_count=$MDX_COUNT" >> "$GITHUB_OUTPUT" + if [ "$MDX_COUNT" -gt 0 ]; then + echo "mdx_changed=true" >> "$GITHUB_OUTPUT" + else + echo "mdx_changed=false" >> "$GITHUB_OUTPUT" + fi + echo "Changed .mdx files: $MDX_COUNT" + + if [ "$MDX_ANY" -gt 0 ]; then + echo "mdx_changed_any=true" >> "$GITHUB_OUTPUT" + else + echo "mdx_changed_any=false" >> "$GITHUB_OUTPUT" + fi + echo "Changed .mdx files including deletions: $MDX_ANY" + + - name: No Mintlify-relevant changes — skipping all checks + if: steps.scope.outputs.mintlify_relevant == 'false' + run: | + echo "Check passes: no Mintlify-relevant file or validation code changes in this PR." + + # ------------------------------------------------------------ + # 2. MDX validation (BLOCKING) — from the old validate-mdx.yml. + # ------------------------------------------------------------ + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7 + if: steps.scope.outputs.mintlify_relevant == 'true' + with: + node-version: '20' + + - name: Get npm paths and cache key + id: npm-config + if: steps.scope.outputs.mintlify_relevant == 'true' + run: | + { + # Force base 10 so a zero-padded day of year such as 008 is not parsed as octal. + echo "date=$(date +%Y)-$(( 10#$(date +%j) / 4 ))" + echo "npm_prefix=$(npm config get prefix)" + echo "npm_cache=$(npm config get cache)" + } >> "$GITHUB_OUTPUT" + + - name: Cache Mintlify CLI + id: cache-mint + if: steps.scope.outputs.mintlify_relevant == 'true' + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 + with: + path: | + ${{ steps.npm-config.outputs.npm_cache }} + ${{ steps.npm-config.outputs.npm_prefix }}/lib/node_modules/mint + ${{ steps.npm-config.outputs.npm_prefix }}/bin/mint + # 4-day cache - expires every 4 days to get latest mint version + key: ${{ runner.os }}-mint-${{ steps.npm-config.outputs.date }} + restore-keys: | + ${{ runner.os }}-mint- + + - name: Install Mintlify CLI (latest) + if: steps.scope.outputs.mintlify_relevant == 'true' && steps.cache-mint.outputs.cache-hit != 'true' + run: npm install -g mint@latest --loglevel=error --no-fund --no-audit + + - name: Verify Mintlify CLI (from cache) + if: steps.scope.outputs.mintlify_relevant == 'true' && steps.cache-mint.outputs.cache-hit == 'true' + run: | + echo "Using cached Mintlify CLI" + mint --version + + - name: Validate MDX with Mintlify + id: validate + if: steps.scope.outputs.mintlify_relevant == 'true' + # Do not fail yet: the comment step below reports the result first, then + # the final verdict step fails the job iff this step failed. + continue-on-error: true + env: + PR_BASE_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || '' }} + PR_HEAD_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || '' }} + run: | + set -o pipefail + ./scripts/mdx-validation/validate-mdx-mintlify.sh 2>&1 | tee validate-output.log + + # ------------------------------------------------------------ + # 3. External link check (informational) — from the old linkcheck-pr.yml. + # ------------------------------------------------------------ + # This check covers external URLs. Internal links are owned by + # `mint broken-links` above, which resolves them against the real route + # table and so knows about docs.json navigation and redirects; lychee + # would resolve them against the repository tree, where a link through a + # redirect has no matching file and reports as broken. lychee.toml excludes + # ^file:// to leave internal linking to Mintlify. The resolution flags below + # stay so local paths still parse into file:// URIs and get excluded quietly + # rather than raising InvalidPathToUri warnings. + # Do not move the ^file:// pattern to a --exclude flag here: a CLI --exclude + # replaces the config's exclude list rather than adding to it, which would + # re-enable every excluded external domain. + # Inputs come from lychee-pr-inputs.txt (written by the scope step) so we + # do not pass every path on the shell argv (ARG_MAX on large PRs). + - name: Check external links (lychee) + id: lychee + if: steps.scope.outputs.mdx_changed == 'true' + timeout-minutes: 10 + # Informational: a lychee crash must not block the PR or the comment. + continue-on-error: true + uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2 + with: + fail: false + # Don't fail if no files to check + failIfEmpty: false + # Output format for reports + format: markdown + # GitHub token for API rate limiting + token: ${{ secrets.GITHUB_TOKEN }} + args: >- + --config lychee.toml + --root-dir ${{ github.workspace }} + --fallback-extensions mdx + --index-files index.mdx,. + --files-from lychee-pr-inputs.txt + + # ------------------------------------------------------------ + # 4. Readability delta (informational) — from the old readability-delta.yml. + # Skips on fork PRs: no access to the private docs-skills submodule or + # W&B Inference. Every step is non-fatal. + # ------------------------------------------------------------ + # The docs-skills submodule is a private repo in another org. Initialize it + # with DOCENGINE_TOKEN so the main checkout token needs no extra scope. + - name: Check out docs-skills submodule + # Non-fatal: a transient auth/network failure here must not fail the PR. + # The report step below detects the missing analyzer and renders the + # informational fallback instead. + continue-on-error: true + if: steps.scope.outputs.is_fork != 'true' && steps.scope.outputs.mdx_changed_any == 'true' + env: + DOCENGINE_TOKEN: ${{ secrets.DOCENGINE_TOKEN }} + run: | + if [ -z "${DOCENGINE_TOKEN:-}" ]; then + echo "::notice::Readability check: DOCENGINE_TOKEN not set; skipping docs-skills submodule checkout." + exit 0 + fi + # One-shot url rewrite so the token is never written to the runner's + # global git config, where later steps (which don't get the secret in + # their env) could recover it from ~/.gitconfig. + # shellcheck disable=SC2140 # intentional concatenation, carried over from the deleted readability-delta.yml + git -c url."https://x-access-token:${DOCENGINE_TOKEN}@github.com/".insteadOf="https://github.com/" \ + submodule update --init --recursive .claude + + - name: Set up Python 3.11 + # Non-fatal: a setup outage in the informational readability path must + # not fail the required job. The report step below fails gracefully + # and renders its fallback note instead. + continue-on-error: true + if: steps.scope.outputs.is_fork != 'true' && steps.scope.outputs.mdx_changed_any == 'true' + uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6 + with: + python-version: "3.11" + + - name: Cache pip packages + # Non-fatal: same reasoning as the setup-python step above. + continue-on-error: true + if: steps.scope.outputs.is_fork != 'true' && steps.scope.outputs.mdx_changed_any == 'true' + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 + with: + path: ~/.cache/pip + key: ${{ runner.os }}-pip-readability-${{ hashFiles('scripts/readability/requirements.txt') }} + restore-keys: | + ${{ runner.os }}-pip-readability- + ${{ runner.os }}-pip- + + - name: Install readability dependencies + # Non-fatal: if deps fail to install, the report step still runs and + # falls back to the informational notice rather than failing the PR. + continue-on-error: true + if: steps.scope.outputs.is_fork != 'true' && steps.scope.outputs.mdx_changed_any == 'true' + run: | + pip install textstat==0.7.13 + pip install -r scripts/readability/requirements.txt || true + + - name: Build readability report + id: readability + if: steps.scope.outputs.is_fork != 'true' && steps.scope.outputs.mdx_changed_any == 'true' + timeout-minutes: 10 + # Non-fatal: a readability crash never blocks the PR. + continue-on-error: true + env: + # Optional: enables the AI agent comprehension judge via W&B Inference. + # The scorer reads WANDB_API_KEY; the repo secret is named + # WANDB_DOCS_INFERENCE_API_KEY (a W&B key with Inference credits). + WANDB_API_KEY: ${{ secrets.WANDB_DOCS_INFERENCE_API_KEY }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + BASE_SHA: ${{ steps.scope.outputs.merge_base }} + HEAD_SHA: ${{ steps.scope.outputs.head_sha }} + run: | + set -euo pipefail + JUDGE_FLAG="" + if [ -n "${WANDB_API_KEY:-}" ]; then + JUDGE_FLAG="--judge" + else + echo "::notice::AI agent comprehension judge skipped: WANDB_DOCS_INFERENCE_API_KEY secret is not set." + fi + # No --include-marker: the combined comment below carries the single + # marker for upserts. + if python scripts/readability/pr_report.py \ + --base "${BASE_SHA}" \ + --head "${HEAD_SHA}" \ + --run-url "${RUN_URL}" \ + --run-id "${GITHUB_RUN_ID}" \ + ${JUDGE_FLAG} \ + > readability-body.md 2> readability-errors.log; then + # Judge failures are non-fatal and land on stderr; show them in the + # step log even on success so they are not silently discarded. + if [ -s readability-errors.log ]; then + echo "Diagnostics from pr_report.py (non-fatal):" + cat readability-errors.log + fi + else + echo "::warning::Readability report failed to build; see logs." + cat readability-errors.log || true + { + echo "## Readability impact" + echo "" + echo "The readability check could not run for this push. This is informational and does not block the PR." + } > readability-body.md + fi + + # ------------------------------------------------------------ + # 5. One combined report: status table + collapsed details. + # ------------------------------------------------------------ + # !cancelled() (not always()) so a run superseded by cancel-in-progress + # does not race the replacement run's comment write. + - name: Assemble doc-quality report + id: report + if: ${{ !cancelled() && steps.scope.outputs.mintlify_relevant == 'true' }} + continue-on-error: true + env: + VALIDATE_OUTCOME: ${{ steps.validate.outcome }} + LYCHEE_OUTCOME: ${{ steps.lychee.outcome }} + LYCHEE_EXIT: ${{ steps.lychee.outputs.exit_code }} + READABILITY_OUTCOME: ${{ steps.readability.outcome }} + IS_FORK: ${{ steps.scope.outputs.is_fork }} + MDX_COUNT: ${{ steps.scope.outputs.mdx_count }} + MDX_CHANGED: ${{ steps.scope.outputs.mdx_changed }} + MDX_CHANGED_ANY: ${{ steps.scope.outputs.mdx_changed_any }} + HEAD_SHA: ${{ steps.scope.outputs.head_sha }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + set -euo pipefail + BODY="comment-body.md" + MAX_SECTION=15000 + + # Print a file capped at MAX_SECTION bytes, keeping the start. + cap_head() { + if [ "$(wc -c < "$1")" -gt "$MAX_SECTION" ]; then + head -c "$MAX_SECTION" "$1" + printf '\n\n_Output truncated; see the [workflow run](%s) for the rest._\n' "$RUN_URL" + else + cat "$1" + fi + } + # Same, keeping the end (validation errors print last). + cap_tail() { + if [ "$(wc -c < "$1")" -gt "$MAX_SECTION" ]; then + printf '_Output truncated; see the [workflow run](%s) for the rest._\n\n' "$RUN_URL" + tail -c "$MAX_SECTION" "$1" + else + cat "$1" + fi + } + + case "$VALIDATE_OUTCOME" in + success) VALIDATE_STATUS="✅ Pass" ;; + failure) VALIDATE_STATUS="❌ **Fail** — blocks this PR" ;; + *) VALIDATE_STATUS="⚠️ Did not run (an earlier step failed — see the [workflow run]($RUN_URL))" ;; + esac + + if [ "$LYCHEE_OUTCOME" = "success" ] && [ "${LYCHEE_EXIT:-0}" = "0" ]; then + LINKS_STATUS="✅ No broken external links (${MDX_COUNT} changed .mdx file(s) checked)" + elif [ "$LYCHEE_OUTCOME" = "success" ]; then + LINKS_STATUS="⚠️ Issues found (informational — see details)" + elif [ "$LYCHEE_OUTCOME" = "failure" ]; then + LINKS_STATUS="⚠️ Link checker did not complete (informational — see the [workflow run]($RUN_URL))" + elif [ "$MDX_CHANGED" = "true" ]; then + # .mdx files DID change but lychee never ran: a blocking setup step + # failed and the implicit success() skipped it. Don't mislabel that + # as "no .mdx changes". + LINKS_STATUS="⚠️ Did not run (an earlier step failed — see the [workflow run]($RUN_URL))" + else + LINKS_STATUS="⏭️ Skipped (no .mdx changes)" + fi + + if [ "$IS_FORK" = "true" ]; then + READ_STATUS="⏭️ Skipped (fork PRs cannot reach the scoring dependencies)" + elif [ "$READABILITY_OUTCOME" = "skipped" ] || [ -z "$READABILITY_OUTCOME" ]; then + if [ "$MDX_CHANGED_ANY" = "true" ]; then + # Same distinction as the link check above. + READ_STATUS="⚠️ Did not run (an earlier step failed — see the [workflow run]($RUN_URL))" + else + READ_STATUS="⏭️ Skipped (no .mdx changes)" + fi + elif [ -s readability-body.md ]; then + READ_STATUS=$(grep -m1 -E '^(Word-weighted|No |The readability)' readability-body.md || echo "See details") + else + READ_STATUS="⚠️ Could not run (informational)" + fi + + { + echo '' + echo '' + echo '## Doc quality' + echo '' + echo '| Check | Result |' + echo '|-------|--------|' + echo "| MDX validation (blocking) | ${VALIDATE_STATUS} |" + echo "| External links (informational) | ${LINKS_STATUS} |" + echo "| Readability (informational) | ${READ_STATUS} |" + echo '' + } > "$BODY" + + # MDX validation details: error excerpt only on failure. + if [ "$VALIDATE_OUTCOME" = "failure" ] && [ -s validate-output.log ]; then + cap_tail validate-output.log > validate-excerpt.txt + # mint output can quote .mdx source containing code fences, and + # tail -c cuts at an arbitrary byte, so a ``` can land at column 0 + # inside the excerpt and close a three-backtick fence early. Size + # the fence one longer than the longest backtick run in the excerpt. + MAX_TICKS=$({ grep -o '`\{3,\}' validate-excerpt.txt || true; } \ + | awk '{ if (length($0) > m) m = length($0) } END { print m + 0 }') + FENCE_LEN=$(( MAX_TICKS >= 3 ? MAX_TICKS + 1 : 3 )) + FENCE=$(printf '%*s' "$FENCE_LEN" '' | tr ' ' '`') + { + echo '
' + echo 'MDX validation errors' + echo '' + echo "${FENCE}text" + cat validate-excerpt.txt + echo "$FENCE" + echo '
' + echo '' + } >> "$BODY" + fi + + # External links details (only when lychee ran and found issues). + if { [ "$LYCHEE_OUTCOME" = "success" ] || [ "$LYCHEE_OUTCOME" = "failure" ]; } \ + && [ "${LYCHEE_EXIT:-0}" != "0" ] && [ -s ./lychee/out.md ]; then + { + echo '
' + echo 'External link check (lychee)' + echo '' + cap_head ./lychee/out.md + echo '' + echo '
' + echo '' + } >> "$BODY" + fi + + # Readability details. + if [ -s readability-body.md ]; then + { + echo '
' + echo 'Readability delta' + echo '' + cap_head readability-body.md + echo '' + echo '
' + echo '' + } >> "$BODY" + fi + + printf 'Updated %s for %s\n' \ + "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" "$HEAD_SHA" >> "$BODY" + + - name: Upsert doc-quality report on PR + # Non-fatal: a transient gh api/comment failure (rate limit, network, + # permissions) must not fail the PR. Fork PRs skip (read-only token). + continue-on-error: true + if: ${{ !cancelled() && steps.report.outcome == 'success' && github.event_name == 'pull_request' && steps.scope.outputs.is_fork != 'true' }} + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + # pipefail so a transient listing failure fails the step (kept green + # by continue-on-error) instead of falling through with an empty + # COMMENT_ID and creating a permanent duplicate comment. + set -euo pipefail + MARKER="" + # Match on author as well as marker so a contributor comment that + # quotes the marker can never be overwritten. Capture the listing + # before taking the first line: `... | head -n1` would close the + # pipe early and SIGPIPE `gh api --paginate` under pipefail. + MATCHES=$(gh api \ + "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \ + --paginate -q ".[] | select((.body | contains(\"${MARKER}\")) and .user.login == \"github-actions[bot]\") | .id") + COMMENT_ID=$(printf '%s\n' "$MATCHES" | head -n1) + + if [ -n "${COMMENT_ID}" ]; then + gh api \ + "repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \ + -X PATCH -F "body=@comment-body.md" + echo "Updated existing doc-quality report comment ${COMMENT_ID}" + else + gh pr comment "${PR_NUMBER}" --body-file comment-body.md + echo "Created new doc-quality report comment" + fi + + - name: Write doc-quality report to job summary + # Fork PRs and non-PR events cannot (or should not) comment; surface the + # report in the workflow step summary instead. + continue-on-error: true + if: ${{ !cancelled() && steps.report.outcome == 'success' && (github.event_name != 'pull_request' || steps.scope.outputs.is_fork == 'true') }} + run: cat comment-body.md >> "${GITHUB_STEP_SUMMARY}" + + # If a later push removes every Mintlify-relevant change, the checks and + # the report steps above all skip — but an earlier push may have left a + # sticky comment, possibly still showing failures. Update it in place to + # an all-skipped state. Never CREATE a comment on this path: PRs that + # never touched relevant files stay comment-free. + - name: Refresh stale report when nothing relevant changed + continue-on-error: true + if: ${{ !cancelled() && steps.scope.outputs.mintlify_relevant == 'false' && github.event_name == 'pull_request' && steps.scope.outputs.is_fork != 'true' }} + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ steps.scope.outputs.head_sha }} + run: | + # Same listing discipline as the upsert step above: fail (kept green + # by continue-on-error) on listing errors rather than misreading + # them as "no comment", and avoid piping --paginate into head. + set -euo pipefail + MARKER="" + MATCHES=$(gh api \ + "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \ + --paginate -q ".[] | select((.body | contains(\"${MARKER}\")) and .user.login == \"github-actions[bot]\") | .id") + COMMENT_ID=$(printf '%s\n' "$MATCHES" | head -n1) + if [ -z "${COMMENT_ID}" ]; then + echo "No existing doc-quality comment; nothing to refresh." + exit 0 + fi + { + echo "${MARKER}" + echo '' + echo '## Doc quality' + echo '' + echo 'This push has no Mintlify-relevant changes, so all doc-quality checks were skipped.' + echo '' + printf 'Updated %s for %s\n' \ + "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" "$HEAD_SHA" + } > comment-body.md + gh api \ + "repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \ + -X PATCH -F "body=@comment-body.md" + echo "Refreshed stale doc-quality report comment ${COMMENT_ID}" + + # One-time migration cleanup: the old linkcheck-pr.yml and + # readability-delta.yml workflows left per-check comments on open PRs. + # Delete them so the single doc-quality comment is the only report. + # Safe to remove this step once all PRs opened before the consolidation + # have been merged or closed. + - name: Clean up legacy per-check comments + continue-on-error: true + if: ${{ !cancelled() && github.event_name == 'pull_request' && steps.scope.outputs.is_fork != 'true' }} + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ github.event.pull_request.number }} + PR_CREATED_AT: ${{ github.event.pull_request.created_at }} + run: | + # Only PRs opened while the replaced workflows still existed can + # carry their comments. Skip the (paginated) listings for PRs opened + # after the consolidation landed, so this step self-retires as the + # pre-consolidation PRs drain. ISO-8601 UTC timestamps compare + # lexicographically. + if [[ "${PR_CREATED_AT}" > "2026-10-01T00:00:00Z" ]]; then + echo "PR opened ${PR_CREATED_AT}, after the doc-quality consolidation; no legacy comments possible." + exit 0 + fi + # Match on author as well as marker so a contributor comment that + # quotes a legacy marker is never deleted. + for MARKER in 'lychee-link-checker-comment' 'readability-delta-report'; do + COMMENT_IDS=$(gh api \ + "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \ + --paginate -q ".[] | select((.body | contains(\"\")) and .user.login == \"github-actions[bot]\") | .id" || true) + for ID in $COMMENT_IDS; do + if gh api "repos/${{ github.repository }}/issues/comments/${ID}" -X DELETE; then + echo "Deleted legacy ${MARKER} comment ${ID}" + fi + done + done + + # ------------------------------------------------------------ + # 6. Verdict: only MDX validation is blocking. + # ------------------------------------------------------------ + - name: Verdict + if: ${{ !cancelled() }} + env: + VALIDATE_OUTCOME: ${{ steps.validate.outcome }} + run: | + if [ "$VALIDATE_OUTCOME" = "failure" ]; then + echo "::error::MDX validation failed (mint validate / mint broken-links). See the 'Validate MDX with Mintlify' step above." + exit 1 + fi + echo "MDX validation: ${VALIDATE_OUTCOME:-skipped}. Link check and readability are informational." diff --git a/.github/workflows/linkcheck-pr.yml b/.github/workflows/linkcheck-pr.yml deleted file mode 100644 index 659c4a9d3f..0000000000 --- a/.github/workflows/linkcheck-pr.yml +++ /dev/null @@ -1,456 +0,0 @@ -name: Link checker - PR changed files - -# Avoid collisions by ensuring only one run per ref -concurrency: - group: linkcheck-pr-${{ github.ref_name }} - cancel-in-progress: false - -on: - workflow_dispatch: - deployment_status: - pull_request: - types: [opened, synchronize, reopened] - # No paths filter - we check paths inside the job for fork PRs only - # This prevents "Skipped" status for same-repo PRs - -permissions: - contents: read - deployments: read - pull-requests: write - -jobs: - linkChecker: - runs-on: ubuntu-latest - - # Run on: - # 1. Manual trigger (workflow_dispatch) - # 2. Successful Mintlify deployment (for same-repo PRs - waits for preview URL) - # 3. PR events from forks only (no Mintlify preview, checks against production) - if: | - github.event_name == 'workflow_dispatch' || - (github.event_name == 'deployment_status' && - github.event.deployment_status.state == 'success' && - github.event.deployment.environment == 'staging' && - contains(github.event.deployment_status.creator.login, 'mintlify') && - contains(github.event.deployment_status.environment_url, 'mintlify')) || - (github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true) - - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - with: - # Needed to diff base..head for the associated PR - fetch-depth: 0 - - - name: Early exit if no docs changed (fork PRs only) - if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true - id: fork-docs-check - uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 - with: - files: | - **/*.mdx - - - name: Report no changes (fork PRs only) - if: | - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.fork-docs-check.outputs.any_changed != 'true' - run: | - echo "✓ No Mintlify page files (.mdx) changed in this fork PR." - echo "Link check not needed - exiting successfully." - - - name: Resolve PR and deployment URL - id: pr-context - if: github.event_name == 'deployment_status' - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 - with: - script: | - const { owner, repo } = context.repo; - const sha = context.payload.deployment?.sha; - const ref = context.payload.deployment?.ref; - const deployUrl = - context.payload.deployment_status?.environment_url || - context.payload.deployment_status?.target_url || - ''; - - core.info(`Deployment SHA: ${sha}`); - core.info(`Deployment ref: ${ref}`); - core.info(`Deployment URL: ${deployUrl}`); - - // Find PR(s) associated with this deployment commit SHA - const prsResp = await github.rest.repos.listPullRequestsAssociatedWithCommit({ - owner, - repo, - commit_sha: sha, - }); - - // A commit can belong to several PRs at once (e.g. one PR branch merged - // into another), and this endpoint's ordering is not stable over time, so - // data[0] can bind the run to an unrelated PR. Match the deployment's own - // branch instead. There is deliberately no data[0] fallback: picking an - // arbitrary PR is the failure mode we are removing, so prefer to skip. - const prs = prsResp.data ?? []; - const pr = - prs.find((p) => p.head?.ref === ref && p.head?.repo?.fork !== true) ?? - prs.find((p) => p.head?.sha === sha) ?? - null; - - if (!pr) { - core.warning(`No PR matched deployment ref '${ref}' at commit ${sha} (${prs.length} PR(s) contain this commit). Skipping linkcheck + PR comment.`); - core.setOutput('pr_number', ''); - core.setOutput('pr_closed', 'false'); - core.setOutput('deploy_url', deployUrl); - return; - } - - // Check if PR is already merged/closed - preview likely torn down - if (pr.state === 'closed') { - core.warning(`PR #${pr.number} is already ${pr.merged_at ? 'merged' : 'closed'}. Preview likely torn down. Skipping link check.`); - core.setOutput('pr_number', ''); - core.setOutput('pr_closed', 'true'); - core.setOutput('deploy_url', ''); - return; - } - - core.info(`Associated PR: #${pr.number} (${pr.html_url})`); - core.setOutput('pr_number', String(pr.number)); - core.setOutput('pr_closed', 'false'); - core.setOutput('base_sha', pr.base.sha); - // Diff the commit Mintlify actually built and is serving at the preview - // URL, not the PR tip: the branch can move between the deploy and this - // run, and pr.head.sha may name a commit that is no longer fetchable. - core.setOutput('head_sha', sha); - core.setOutput('deploy_url', deployUrl); - - - name: Resolve GitHub-style PR diff - id: pr-diff - if: | - (github.event_name == 'deployment_status' && - steps.pr-context.outputs.pr_number != '') || - (github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true) - env: - TARGET_SHA: ${{ steps.pr-context.outputs.base_sha || github.event.pull_request.base.sha }} - HEAD_SHA: ${{ steps.pr-context.outputs.head_sha || github.event.pull_request.head.sha }} - run: | - set -euo pipefail - - # Skip rather than fail: a link check that cannot resolve its own refs is an - # infrastructure hiccup, and failing here puts a red X on an unrelated docs PR. - missing="" - for sha_name in TARGET_SHA HEAD_SHA; do - sha_value="${!sha_name}" - if [ -z "$sha_value" ] || ! git cat-file -e "${sha_value}^{commit}" 2>/dev/null; then - missing="$missing $sha_name=${sha_value:-}" - fi - done - - if [ -n "$missing" ]; then - echo "::warning::Link check skipped - commit(s) not in this checkout:$missing (branch force-pushed or deleted since the deployment?)" - echo "skip=true" >> "$GITHUB_OUTPUT" - exit 0 - fi - - if ! merge_base_sha=$(git merge-base "$TARGET_SHA" "$HEAD_SHA"); then - echo "::warning::Link check skipped - no merge base between target $TARGET_SHA and head $HEAD_SHA" - echo "skip=true" >> "$GITHUB_OUTPUT" - exit 0 - fi - - echo "skip=false" >> "$GITHUB_OUTPUT" - echo "merge_base_sha=$merge_base_sha" >> "$GITHUB_OUTPUT" - echo "head_sha=$HEAD_SHA" >> "$GITHUB_OUTPUT" - echo "PR diff: $merge_base_sha...$HEAD_SHA (target tip: $TARGET_SHA)" - - - name: Get changed documentation files - id: changed-files - # Skip this for fork PRs with no doc changes (already checked above) - # For deployment_status and workflow_dispatch, always run - if: | - steps.pr-diff.outputs.skip != 'true' && - ((github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.fork-docs-check.outputs.any_changed == 'true') || - (github.event_name == 'deployment_status' && - steps.pr-context.outputs.pr_number != '') || - github.event_name == 'workflow_dispatch') - uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 - with: - base_sha: ${{ steps.pr-diff.outputs.merge_base_sha }} - sha: ${{ steps.pr-diff.outputs.head_sha }} - files: | - **/*.mdx - - - name: Get PR number - id: get-pr - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 - with: - script: | - // For pull_request events, get PR number directly - if (context.eventName === 'pull_request') { - const prNumber = context.payload.pull_request.number; - core.info(`PR number from pull_request event: ${prNumber}`); - core.setOutput('pr_number', prNumber); - return prNumber; - } - - // For deployment_status events, reuse the PR that "Resolve PR and - // deployment URL" already resolved. Repeating the lookup here let the two - // steps disagree, so the diff could come from one PR while the results - // comment landed on another. - if (context.eventName === 'deployment_status') { - const prNumber = '${{ steps.pr-context.outputs.pr_number }}'; - if (!prNumber) { - core.warning('No PR resolved for this deployment. Skipping PR comment.'); - return null; - } - core.info(`PR number from pr-context step: ${prNumber}`); - core.setOutput('pr_number', prNumber); - return prNumber; - } - - core.warning(`Unsupported event type: ${context.eventName}`); - return null; - - - name: Clear stale comment when no documentation changed - # Guard on pr-diff skip too: when the diff could not be resolved, changed-files - # never ran, and an unset any_changed must not be reported as "no .mdx changed". - if: | - github.event_name == 'deployment_status' && - steps.pr-context.outputs.pr_number != '' && - steps.pr-diff.outputs.skip != 'true' && - steps.changed-files.outputs.any_changed != 'true' - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 - with: - script: | - const identifier = ''; - const prNumber = Number('${{ steps.pr-context.outputs.pr_number }}'); - const { data: comments } = await github.rest.issues.listComments({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: prNumber, - }); - - const existingComment = comments.find(comment => - comment.body?.includes(identifier) && comment.user?.login === 'github-actions[bot]' - ); - - if (!existingComment) { - core.info(`No existing link-check comment on PR #${prNumber}.`); - return; - } - - await github.rest.issues.updateComment({ - owner: context.repo.owner, - repo: context.repo.repo, - comment_id: existingComment.id, - body: `${identifier}\n## 🔗 Link Checker Results\n\n✅ **No documentation links to check**\n\nThis PR does not change any \`.mdx\` files.`, - }); - core.info(`Cleared stale link-check results on PR #${prNumber}.`); - - - name: Debug - Show lychee config being used - if: | - steps.pr-context.outputs.pr_closed != 'true' && - steps.pr-diff.outputs.skip != 'true' && - ((github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.fork-docs-check.outputs.any_changed == 'true') || - (github.event_name == 'deployment_status' && - steps.changed-files.outputs.any_changed == 'true') || - github.event_name == 'workflow_dispatch') - run: | - echo "=== Lychee Configuration Debug ===" - echo "Working directory: $(pwd)" - echo "Git HEAD commit: $(git rev-parse HEAD)" - echo "Git branch: $(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo 'detached HEAD')" - echo "" - if [ -f "lychee.toml" ]; then - echo "✓ lychee.toml found in working directory" - echo "Last modified by commit: $(git log -1 --format='%H %ai %s' -- lychee.toml)" - echo "" - echo "=== Current verbosity setting ===" - grep "^verbose = " lychee.toml || echo "No verbose setting found" - echo "" - echo "=== Last 5 exclude patterns ===" - grep -A 25 "^exclude = " lychee.toml | tail -6 - else - echo "✗ lychee.toml not found in working directory!" - fi - if [ -f lychee-pr-inputs.txt ]; then - echo "" - echo "=== Lychee input paths (from --files-from) ===" - wc -l lychee-pr-inputs.txt - fi - echo "==================================" - - - name: Write lychee input file list - id: lychee-inputs-file - # Same conditions as Link Checker. Use git diff to write paths instead of - # passing thousands of files through the workflow expression into args - # (avoids: Argument list too long / ARG_MAX). - if: | - steps.pr-context.outputs.pr_closed != 'true' && - steps.pr-diff.outputs.skip != 'true' && - ((github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.fork-docs-check.outputs.any_changed == 'true') || - (github.event_name == 'deployment_status' && - steps.changed-files.outputs.any_changed == 'true') || - github.event_name == 'workflow_dispatch') - env: - EVENT_NAME: ${{ github.event_name }} - BASE_SHA: ${{ steps.pr-diff.outputs.merge_base_sha }} - HEAD_SHA: ${{ steps.pr-diff.outputs.head_sha }} - DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - run: | - set -euo pipefail - OUT="lychee-pr-inputs.txt" - : > "$OUT" - - filter_to_docs() { - grep -E '\.mdx$' || true - } - - if [ "$EVENT_NAME" = "workflow_dispatch" ]; then - base=$(git merge-base HEAD "origin/${DEFAULT_BRANCH}") - git diff --name-only --diff-filter=d "$base" HEAD | filter_to_docs > "$OUT" - elif [ -n "$BASE_SHA" ] && [ -n "$HEAD_SHA" ]; then - git diff --name-only --diff-filter=d "$BASE_SHA" "$HEAD_SHA" | filter_to_docs > "$OUT" - else - echo "linkcheck-pr: could not determine base/head refs for input list" >&2 - exit 1 - fi - - # Match previous args fallback: workflow_dispatch used '.' when no paths matched - if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ ! -s "$OUT" ]; then - echo '.' > "$OUT" - fi - - echo "lychee-pr-inputs.txt: $(wc -l < "$OUT" | tr -d ' ') path(s)" - - - name: Link Checker - id: lychee - # Run if: - # - Fork PR with doc changes - # - Deployment status event with doc changes (and PR not closed) - # - Manual workflow dispatch - if: | - steps.pr-context.outputs.pr_closed != 'true' && - steps.pr-diff.outputs.skip != 'true' && - ((github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.fork-docs-check.outputs.any_changed == 'true') || - (github.event_name == 'deployment_status' && - steps.changed-files.outputs.any_changed == 'true') || - github.event_name == 'workflow_dispatch') - uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2 - with: - fail: false - # Don't fail if no files to check - failIfEmpty: false - # Output format for reports - format: markdown - # GitHub token for API rate limiting - token: ${{ secrets.GITHUB_TOKEN }} - # This check covers external URLs. Internal links are owned by - # `mint broken-links` in validate-mdx, which resolves them against the real - # route table and so knows about docs.json navigation and redirects; lychee - # would resolve them against the repository tree, where a link through a - # redirect has no matching file and reports as broken. lychee.toml excludes - # ^file:// to leave internal linking to Mintlify. The resolution flags below - # stay so local paths still parse into file:// URIs and get excluded quietly - # rather than raising InvalidPathToUri warnings. - # Do not move the ^file:// pattern to a --exclude flag here: a CLI --exclude - # replaces the config's exclude list rather than adding to it, which would - # re-enable every excluded external domain. - # Inputs come from lychee-pr-inputs.txt (see "Write lychee input file list") so - # we do not pass every path on the shell argv (ARG_MAX on large PRs). - args: >- - --config lychee.toml - --root-dir ${{ github.workspace }} - --fallback-extensions mdx - --index-files index.mdx,. - --files-from lychee-pr-inputs.txt - - - name: Skip comment for fork PR - if: | - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.fork == true && - steps.get-pr.outputs.pr_number && - (steps.lychee.conclusion == 'success' || - steps.lychee.conclusion == 'failure') - run: | - echo "ℹ️ Skipping PR comment for fork PR #${{ steps.get-pr.outputs.pr_number }}" - echo "Fork PRs have restricted GITHUB_TOKEN permissions and cannot write comments." - echo "Link check completed successfully - see logs above for any issues." - - - name: Comment on PR with results - # Comment if link checker ran and not a fork PR - # Skip for fork PRs to avoid "Resource not accessible by integration" error - if: | - steps.get-pr.outputs.pr_number && - (steps.lychee.conclusion == 'success' || - steps.lychee.conclusion == 'failure') && - (github.event_name != 'pull_request' || - github.event.pull_request.head.repo.fork != true) - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 - with: - script: | - const fs = require('fs'); - const prNumber = parseInt('${{ steps.get-pr.outputs.pr_number }}'); - const exitCode = parseInt('${{ steps.lychee.outputs.exit_code }}') || 0; - const deployUrl = '${{ steps.pr-context.outputs.deploy_url || github.event.deployment_status.environment_url }}'; - - const identifier = ''; - let commentBody = identifier + '\n'; - const previewContext = deployUrl ? `\n_Preview: ${deployUrl}_\n` : ''; - - if (exitCode === 0) { - // Success - no broken links - commentBody += '## 🔗 Link Checker Results\n\n'; - commentBody += '✅ **All links are valid!**\n\n'; - commentBody += 'No broken links were detected.\n'; - commentBody += previewContext; - } else { - // Issues found - try { - const report = fs.readFileSync('./lychee/out.md', 'utf8'); - commentBody += '## 🔗 Link Checker Results\n\n'; - commentBody += '⚠️ **Some issues were detected**\n\n'; - commentBody += previewContext; - commentBody += '---\n\n'; - commentBody += report; - } catch (e) { - commentBody += '## 🔗 Link Checker Results\n\n'; - commentBody += '⚠️ Link checker found issues but could not read report.\n'; - } - } - - // Find and update existing comment, or create new one - const { data: comments } = await github.rest.issues.listComments({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: prNumber, - }); - - const existingComment = comments.find(comment => - comment.body?.includes(identifier) && comment.user?.login === 'github-actions[bot]' - ); - - if (existingComment) { - await github.rest.issues.updateComment({ - owner: context.repo.owner, - repo: context.repo.repo, - comment_id: existingComment.id, - body: commentBody - }); - core.info(`Updated comment ${existingComment.id} on PR #${prNumber}`); - } else { - await github.rest.issues.createComment({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: prNumber, - body: commentBody - }); - core.info(`Created new comment on PR #${prNumber}`); - } diff --git a/.github/workflows/readability-delta.yml b/.github/workflows/readability-delta.yml deleted file mode 100644 index 9f04b2944a..0000000000 --- a/.github/workflows/readability-delta.yml +++ /dev/null @@ -1,228 +0,0 @@ -# ------------------------------------------------------------------ -# Readability delta check (DOCS-2626) -# ------------------------------------------------------------------ -# -# What this workflow does -# ----------------------- -# Posts an informational, non-blocking PR comment describing how the PR affects -# the readability of the English docs it touches. The scoring logic lives in the -# coreweave/docs-skills submodule (.claude/scripts/_readability.py and -# _docs_eval_lib.py); scripts/readability/pr_report.py is the wandb/docs glue -# that diffs base..head, scores each changed .mdx file, and builds the Markdown. -# -# This check never fails a PR. Every step exits 0; the worst case is a comment -# noting that scoring was unavailable. -# -# Auth -# ---- -# - Main checkout uses the default GITHUB_TOKEN (the workflow runs in the -# wandb/docs PR context). -# - The coreweave/docs-skills submodule is private and cross-org, so a separate -# step initializes it with DOCENGINE_TOKEN (the same x-access-token credential -# used for the gitsubmodule ecosystem in .github/dependabot.yml). That token -# needs no wandb/docs scope. -# - The AI agent comprehension judge calls W&B Inference with the -# WANDB_DOCS_INFERENCE_API_KEY secret (a W&B key with Inference credits), -# passed to the scorer as WANDB_API_KEY. When that secret is absent the -# deterministic textstat delta still runs. -# -# Forks -# ----- -# Fork PRs have no access to repo secrets, so neither the submodule fetch nor the -# judge can run. The first step detects a fork and makes the whole job a no-op -# (posts an Actions notice, exits 0). Forks are uncommon in wandb/docs and -# coreweave repos cannot use forks at all, so the lost coverage is negligible. -# ------------------------------------------------------------------ - -name: Readability delta - -on: - pull_request: - types: [opened, synchronize, reopened] - paths: - - "**/*.mdx" - workflow_dispatch: - -concurrency: - group: readability-delta-${{ github.ref }} - cancel-in-progress: true - -permissions: - contents: read - pull-requests: write - -jobs: - readability: - name: Report readability delta - runs-on: ubuntu-latest - steps: - # Fork no-op guard: fork PRs cannot reach the private submodule or W&B - # Inference, so skip the whole check (still reports success). - - name: Skip on fork PRs - id: fork - run: | - if [ "${{ github.event_name }}" = "pull_request" ] && \ - [ "${{ github.event.pull_request.head.repo.fork }}" = "true" ]; then - echo "is_fork=true" >> "$GITHUB_OUTPUT" - echo "::notice::Readability check skipped: fork PRs have no access to the docs-skills submodule or W&B Inference." - else - echo "is_fork=false" >> "$GITHUB_OUTPUT" - fi - - - name: Check out repository - if: steps.fork.outputs.is_fork != 'true' - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - with: - # Need full history to diff base..head. - fetch-depth: 0 - submodules: false - - # The docs-skills submodule is a private repo in another org. Initialize it - # with DOCENGINE_TOKEN so the main checkout token needs no extra scope. - - name: Check out docs-skills submodule - # Non-fatal: a transient auth/network failure here must not fail the PR. - # The report step below detects the missing analyzer and posts the - # informational fallback comment instead. - continue-on-error: true - if: steps.fork.outputs.is_fork != 'true' - env: - DOCENGINE_TOKEN: ${{ secrets.DOCENGINE_TOKEN }} - run: | - if [ -z "${DOCENGINE_TOKEN:-}" ]; then - echo "::notice::Readability check: DOCENGINE_TOKEN not set; skipping docs-skills submodule checkout." - exit 0 - fi - # One-shot url rewrite so the token is never written to the runner's - # global git config, where later steps (which don't get the secret in - # their env) could recover it from ~/.gitconfig. - git -c url."https://x-access-token:${DOCENGINE_TOKEN}@github.com/".insteadOf="https://github.com/" \ - submodule update --init --recursive .claude - - - name: Set up Python 3.11 - if: steps.fork.outputs.is_fork != 'true' - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6 - with: - python-version: "3.11" - - - name: Cache pip packages - if: steps.fork.outputs.is_fork != 'true' - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 - with: - path: ~/.cache/pip - key: ${{ runner.os }}-pip-readability-${{ hashFiles('scripts/readability/requirements.txt') }} - restore-keys: | - ${{ runner.os }}-pip-readability- - ${{ runner.os }}-pip- - - - name: Install dependencies - # Non-fatal: if deps fail to install, the report step still runs and - # falls back to the informational notice rather than failing the PR. - continue-on-error: true - if: steps.fork.outputs.is_fork != 'true' - run: | - pip install textstat==0.7.13 - pip install -r scripts/readability/requirements.txt || true - - # Resolve base and head SHAs. For pull_request, use the merge-base of the - # PR's base and head: the event's base.sha is the base branch TIP, so a PR - # branch that is behind the base branch would otherwise diff (and read - # before-text) against unrelated mainline changes. For workflow_dispatch, - # diff the merge-base of the current ref against the default branch. - - name: Resolve base and head - id: refs - if: steps.fork.outputs.is_fork != 'true' - env: - EVENT_NAME: ${{ github.event_name }} - PR_BASE: ${{ github.event.pull_request.base.sha }} - PR_HEAD: ${{ github.event.pull_request.head.sha }} - DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - run: | - set -euo pipefail - if [ "$EVENT_NAME" = "pull_request" ]; then - BASE=$(git merge-base "$PR_BASE" "$PR_HEAD" || echo "$PR_BASE") - echo "base=$BASE" >> "$GITHUB_OUTPUT" - echo "head=$PR_HEAD" >> "$GITHUB_OUTPUT" - else - git fetch --no-tags --depth=200 origin "$DEFAULT_BRANCH" || true - BASE=$(git merge-base "origin/${DEFAULT_BRANCH}" HEAD || git rev-parse HEAD~1) - echo "base=$BASE" >> "$GITHUB_OUTPUT" - echo "head=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" - fi - - - name: Build readability report - id: report - if: steps.fork.outputs.is_fork != 'true' - env: - # Optional: enables the AI agent comprehension judge via W&B Inference. - # The scorer reads WANDB_API_KEY; the repo secret is named - # WANDB_DOCS_INFERENCE_API_KEY (a W&B key with Inference credits). - WANDB_API_KEY: ${{ secrets.WANDB_DOCS_INFERENCE_API_KEY }} - RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - run: | - set -euo pipefail - JUDGE_FLAG="" - if [ -n "${WANDB_API_KEY:-}" ]; then - JUDGE_FLAG="--judge" - else - echo "::notice::AI agent comprehension judge skipped: WANDB_DOCS_INFERENCE_API_KEY secret is not set." - fi - # Never fail the PR: capture failures and fall back to a notice. - if python scripts/readability/pr_report.py \ - --base "${{ steps.refs.outputs.base }}" \ - --head "${{ steps.refs.outputs.head }}" \ - --include-marker \ - --run-url "${RUN_URL}" \ - --run-id "${GITHUB_RUN_ID}" \ - ${JUDGE_FLAG} \ - > comment-body.md 2> report-errors.log; then - echo "ok=true" >> "$GITHUB_OUTPUT" - # Judge failures are non-fatal and land on stderr; show them in the - # step log even on success so they are not silently discarded. - if [ -s report-errors.log ]; then - echo "Diagnostics from pr_report.py (non-fatal):" - cat report-errors.log - fi - else - echo "ok=false" >> "$GITHUB_OUTPUT" - echo "::warning::Readability report failed to build; see logs." - cat report-errors.log || true - { - echo "" - echo "" - echo "## Readability impact" - echo "" - echo "The readability check could not run for this push. This is informational and does not block the PR." - } > comment-body.md - fi - - - name: Upsert readability report on PR - # Non-fatal: a transient gh api/comment failure (rate limit, network, - # permissions) must not fail the PR. The check stays informational. - continue-on-error: true - if: steps.fork.outputs.is_fork != 'true' && github.event_name == 'pull_request' - env: - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ github.event.pull_request.number }} - run: | - MARKER="" - COMMENT_ID=$(gh api \ - "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \ - --paginate -q ".[] | select(.body | contains(\"${MARKER}\")) | .id" \ - | head -n1) - - if [ -n "${COMMENT_ID}" ]; then - gh api \ - "repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \ - -X PATCH -F "body=@comment-body.md" - echo "Updated existing readability report comment ${COMMENT_ID}" - else - gh pr comment "${PR_NUMBER}" --body-file comment-body.md - echo "Created new readability report comment" - fi - - - name: Write readability report to job summary - # Non-fatal: if comment-body.md is missing due to an earlier unexpected - # failure, writing the summary must not fail the otherwise-green job. - continue-on-error: true - if: steps.fork.outputs.is_fork != 'true' && github.event_name == 'workflow_dispatch' - run: cat comment-body.md >> "${GITHUB_STEP_SUMMARY}" diff --git a/.github/workflows/validate-mdx.yml b/.github/workflows/validate-mdx.yml deleted file mode 100644 index 600ed977ff..0000000000 --- a/.github/workflows/validate-mdx.yml +++ /dev/null @@ -1,102 +0,0 @@ -name: Validate MDX -on: - pull_request: - push: - branches: - - main - workflow_dispatch: - -permissions: - contents: read - -concurrency: - group: validate-mdx-${{ github.ref }} - cancel-in-progress: true - -jobs: - validate-mdx: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - with: - fetch-depth: 0 - - # Pushes made with the default workflow GITHUB_TOKEN do not trigger other - # pull_request workflows. Workflows that push with the wandb-docs-pr-writer - # App token (Compress Images, Build CSS, Knowledgebase Nav) avoid that. - # Otherwise branch protection may need a human push or workflow_dispatch. - - name: Determine if Mintlify validation is needed - id: scope - run: | - set -euo pipefail - - if [ "${{ github.event_name }}" != "pull_request" ]; then - echo "needs_validation=true" >> "$GITHUB_OUTPUT" - echo "Runs full validation for push/workflow_dispatch (not a PR diff)." - exit 0 - fi - base="${{ github.event.pull_request.base.sha }}" - head="${{ github.event.pull_request.head.sha }}" - # Pathspecs do not support regex alternation like *.(mdx|json); filter with grep instead. - # Includes .json for docs.json / mint.json / OpenAPI specs that affect generated pages. - # Disable rename detection so deleting or renaming a relevant file still triggers validation. - changed_files=$(git diff --name-only --no-renames "${base}...${head}") - if grep -Eq '(\.(mdx|json|ya?ml|png|jpe?g|webp)$|^scripts/mdx-validation/validate-mdx-mintlify\.sh$)' <<< "$changed_files"; then - echo "needs_validation=true" >> "$GITHUB_OUTPUT" - echo "Found changed Mintlify-relevant files or validation code; will run Mintlify validation." - else - echo "needs_validation=false" >> "$GITHUB_OUTPUT" - echo "No Mintlify-relevant files in this PR diff; skipping Mintlify install and validation." - fi - - - name: No Mintlify-relevant changes — skipping validation - if: steps.scope.outputs.needs_validation == 'false' - run: | - echo "Check passes: no Mintlify-relevant file or validation code changes in this PR." - - - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7 - if: steps.scope.outputs.needs_validation == 'true' - with: - node-version: '20' - - - name: Get npm paths and cache key - id: npm-config - if: steps.scope.outputs.needs_validation == 'true' - run: | - { - # Force base 10 so a zero-padded day of year such as 008 is not parsed as octal. - echo "date=$(date +%Y)-$(( 10#$(date +%j) / 4 ))" - echo "npm_prefix=$(npm config get prefix)" - echo "npm_cache=$(npm config get cache)" - } >> "$GITHUB_OUTPUT" - - - name: Cache Mintlify CLI - id: cache-mint - if: steps.scope.outputs.needs_validation == 'true' - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 - with: - path: | - ${{ steps.npm-config.outputs.npm_cache }} - ${{ steps.npm-config.outputs.npm_prefix }}/lib/node_modules/mint - ${{ steps.npm-config.outputs.npm_prefix }}/bin/mint - # 4-day cache - expires every 4 days to get latest mint version - key: ${{ runner.os }}-mint-${{ steps.npm-config.outputs.date }} - restore-keys: | - ${{ runner.os }}-mint- - - - name: Install Mintlify CLI (latest) - if: steps.scope.outputs.needs_validation == 'true' && steps.cache-mint.outputs.cache-hit != 'true' - run: npm install -g mint@latest --loglevel=error --no-fund --no-audit - - - name: Verify Mintlify CLI (from cache) - if: steps.scope.outputs.needs_validation == 'true' && steps.cache-mint.outputs.cache-hit == 'true' - run: | - echo "Using cached Mintlify CLI" - mint --version - - - name: Validate MDX with Mintlify - if: steps.scope.outputs.needs_validation == 'true' - env: - PR_BASE_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || '' }} - PR_HEAD_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || '' }} - run: ./scripts/mdx-validation/validate-mdx-mintlify.sh diff --git a/scripts/mdx-validation/validate-mdx-mintlify.sh b/scripts/mdx-validation/validate-mdx-mintlify.sh index 0464e3a2b4..bf935d9e15 100755 --- a/scripts/mdx-validation/validate-mdx-mintlify.sh +++ b/scripts/mdx-validation/validate-mdx-mintlify.sh @@ -1,7 +1,8 @@ #!/bin/bash set -euo pipefail -# Match .github/workflows/validate-mdx.yml: pages (.mdx), Mintlify config / OpenAPI (.json, .yaml), +# Keep byte-identical to the scope-gate grep in .github/workflows/doc-quality.yml, which decides +# whether this script runs at all: pages (.mdx), Mintlify config / OpenAPI (.json, .yaml), # common doc assets, and this validation script. OpenAPI and docs.json drive generated MDX at build time. MINTLIFY_RELEVANT_REGEX='(\.(mdx|json|ya?ml|png|jpe?g|webp)$|^scripts/mdx-validation/validate-mdx-mintlify\.sh$)' CHECK=${1:-all} diff --git a/scripts/readability/README.md b/scripts/readability/README.md index d518b78cad..7ed1aba0a8 100644 --- a/scripts/readability/README.md +++ b/scripts/readability/README.md @@ -29,7 +29,8 @@ This directory holds the wandb/docs plumbing: file, aggregates a word-weighted delta, and builds the Markdown report. - `tests/` covers the parsing and Markdown-building logic (no network needed). -The workflow is `.github/workflows/readability-delta.yml`. +The workflow is `.github/workflows/doc-quality.yml`, which runs this check as +its informational readability step. ## Behavior diff --git a/scripts/readability/pr_report.py b/scripts/readability/pr_report.py index b442a5fb0c..acf018f1a1 100644 --- a/scripts/readability/pr_report.py +++ b/scripts/readability/pr_report.py @@ -36,7 +36,7 @@ -------- - ``.claude/scripts/_readability.py`` for the analyzer. - ``.claude/scripts/_docs_eval_lib.py`` for the comprehension judge rubric. -- ``.github/workflows/readability-delta.yml`` for when this runs. +- ``.github/workflows/doc-quality.yml`` for when this runs. """ from __future__ import annotations @@ -54,7 +54,10 @@ # Constants # --------------------------------------------------------------------------- -# GitHub issue comments are upserted by searching for this exact HTML comment. +# Legacy marker from the retired readability-delta.yml workflow; emitted only +# with --include-marker (manual use). CI now posts one combined comment keyed +# by , and its legacy-cleanup step deletes bot +# comments that carry THIS marker. REPORT_MARKER = "" # Visible heading in PR comments and job summaries (sentence case per docs style). @@ -520,7 +523,11 @@ def main() -> None: parser.add_argument("--run-id", default=None, help="Actions run id for link text.") parser.add_argument( "--include-marker", action="store_true", - help="Prefix output with the HTML marker for PR comment upserts.", + help="Prefix output with the legacy readability-delta-report HTML " + "marker. Manual use only: no CI caller passes this flag, and the " + "doc-quality workflow's legacy-comment cleanup DELETES bot PR " + "comments carrying this marker, so a comment posted with it " + "would vanish on the next push.", ) parser.add_argument( "--name-status", default=None,