[robobun]

Problem

• Build lanes die in a dep fetch: error: Failed to download after 5 attempts: https://github.com/oven-sh/lol-html/archive/725ce499....tar.gz, cause: fetch failed. Build 93382 lost linux x64-asan on lolhtml. Build 93392 (a one-file PR) lost four lanes: x64-musl and x64-android on cares, mimalloc and the WebKit tarball, freebsd aarch64 on lolhtml, windows aarch64 on cares, libuv, mimalloc and WebKit.
• Those are the downloads that miss the image's prefetch cache (everything else in the same logs says using prefetch cache): the deps whose pins moved after the images were baked in ci: single arm64 debian-13 build host; ThinLTO everywhere; baseline-only x64; rust+link merge; sysroots; WebKit a36c188; rust 2026-07-20 #34782 (lolhtml HTMLRewriter: stream through SinkHandle/SourceHandle; suspend async handlers instead of wait_for_promise #36733, mimalloc Update mimalloc to the upstream dev3 (v3.4.3) sync #36431, c-ares deps: update c-ares to v1.34.8 #34007, libuv Bump libuv: uv_spawn returns an error instead of aborting on AssignProcessToJobObject failure #36839, WebKit several times a week), plus WebKit on every lane other than linux arm64, because prefetch-deps.ts only enumerates the bake host's own target (handed off separately). Each build makes on the order of a hundred live github.com downloads.
• downloadWithRetry (scripts/build/download.ts:156) made 5 attempts with 2+4+8+16s of backoff, about 30s in total. The logs show agent-wide outages longer than that: on the x64-musl lane three downloads that started together failed on all five attempts over roughly two minutes, after lolhtml had succeeded on the same agent seconds earlier; on the freebsd lane lolhtml failed five times in a row while cares and mimalloc went through and WebKit succeeded on its second try.
• BuildError.format() (scripts/build/error.ts:33) prints one level of cause. node's fetch throws TypeError: fetch failed and keeps the real error (DNS, connect timeout, reset) in .cause, so every one of these logs says only cause: fetch failed, and the retry lines say nothing about what failed.

Fix

• downloadRetry: 10 attempts, backoff doubling from 2s and capped at 30s, 180s of backoff in total instead of 30s. Exported as a RetryPolicy (optional last parameter of downloadWithRetry) so the test can run the production attempt count with the backoff zeroed.
• 408 and 429 are retried along with 5xx and network errors. Other 4xx still fail on the first attempt and are still thrown unwrapped, which prefetch-deps.ts relies on to tell a 404 (variant not published) from a transient failure.
• Each retry line names the failure it is retrying, e.g. retry 2/10 in 2000ms (fetch failed: other side closed), and format() prints the whole cause chain through the new describeError(), so the next one of these says what the network did.
• Why here: the prefetch cache goes stale by design as soon as a pin moves, and WebKit moves faster than images get rebaked (the images were rebaked on July 21 and this came back within two weeks; [publish images] bootstrap: rebake CI images to refresh prefetched deps #30095 was an earlier rebake for the same symptom), so the live path is permanently on every build's critical path and has to outlast the outages CI actually sees. The wider window only costs time while github.com is actually down, when the lane would otherwise have failed; build-bun's step timeout is 60 minutes.
• Verified with test/internal/build-download-retry.test.ts (bun bd test, 7 pass). It drives the real downloadWithRetry against a local server that drops connections or returns scripted statuses, checks the retry lines and format() output, and pins the schedule's total backoff at two minutes or more. Against the previous download.ts/error.ts the file fails at import (downloadRetry and describeError did not exist); each behavioral case is something the old loop did not do (10 attempts, 429 retried, reason on the retry line, cause chain in format()).
• Also ran the loop under node, the runtime CI builds with, against a dropping server: retry lines read (fetch failed: other side closed) and the final report prints cause: fetch failed: other side closed. bunx tsc -p scripts/build/tsconfig.json reports nothing for these files.

Background

• Dep fetching: configure emits one ninja dep_fetch edge per vendored dep, which runs scripts/build/fetch-cli.ts; that calls downloadWithRetry on https://github.com/<repo>/archive/<commit>.tar.gz, and fetchPrebuilt uses the same function for release tarballs such as WebKit. Both URL kinds start with a 302 from github.com (to codeload.github.com and objects.githubusercontent.com respectively), which is why one github.com problem takes out both kinds at once.
• Prefetch cache: CI images run scripts/prefetch-deps.ts at bake time, storing each tarball under /opt/bun-prefetch/by-url/<sha256(url)>. downloadWithRetry looks there before touching the network, so a dep is served from the image only while its pinned URL is the one that was current at bake time; anything bumped later downloads live until the next [publish images] rebake.
• BuildError is the build system's error type; fetch-cli.ts and build.ts print failures through its format(), which produces the error: / hint: / cause: lines seen in the build log.

[coderabbitai]

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 2 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1979945e-931a-4c82-8c0c-f7b13daa6baf

📥 Commits

Reviewing files that changed from the base of the PR and between 9a543cc and 4be662b.

📒 Files selected for processing (3)

• scripts/build/download.ts
• scripts/build/error.ts
• test/internal/build-download-retry.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[robobun]

Status: reproduced from the CI logs of builds 93382 and 93392 (five build lanes across the two builds failed in dep fetches with Failed to download after 5 attempts / cause: fetch failed; in the same logs every prefetched dep says using prefetch cache, and the failing ones are the live downloads). Fix and test are in this PR; bun bd test test/internal/build-download-retry.test.ts passes, and the test file fails against the previous scripts/build/download.ts and error.ts.

[claude]

LGTM — self-contained build-tooling change with solid test coverage.

What was reviewed:

• Retry schedule math (2+4+8+16+30×5 = 180s) and that 404 stays permanent/unwrapped so prefetch-deps.ts's /\bHTTP 404\b/ match still works.
• All downloadWithRetry callers (fetch-cli.ts, winsysroot.ts, prefetch-deps.ts, fetchPrebuilt) pass 3 args, so the new optional retry param is transparent.
• describeError handles cycles, AggregateError, and non-Error causes; tests use a local net server on 127.0.0.1:0 with zero-backoff injection, cleaned up via await using / mockRestore in finally.

Extended reasoning...

Overview

This PR touches only build-system tooling: scripts/build/download.ts (retry schedule extracted into an exported RetryPolicy, 5→10 attempts, 30s→180s total backoff, 408/429 added to the retryable set, retry log lines now include the failure reason), scripts/build/error.ts (new describeError() that walks the full .cause chain including AggregateError, wired into BuildError.format()), and a new test/internal/build-download-retry.test.ts. No runtime code, no user-facing API.

Security risks

None. This is the build system's dep-fetch path downloading from pinned GitHub URLs. The change only adjusts retry timing and error formatting; no new inputs are parsed, no new trust boundaries are crossed.

Level of scrutiny

Low-to-medium. It's build infrastructure, not shipped runtime. The failure mode of the change being wrong is a build lane waiting ~3 minutes instead of ~30 seconds before giving up on a genuinely dead network — well inside the 60-minute step timeout the PR notes. The 404-stays-permanent contract that prefetch-deps.ts and prebuiltDownloadError depend on is preserved (verified: 404 still hits permanent = true and is thrown unwrapped with HTTP 404 ... in the message).

Other factors

• All existing callers (fetch-cli.ts, winsysroot.ts, prefetch-deps.ts, and internal fetchPrebuilt) pass only three positional args, so the new optional fourth parameter defaults to the production schedule and is backward-compatible.
• describeError guards against cycles with a Set, falls back to .name when .message is empty, flattens AggregateError.errors, and appends non-Error causes — all covered by the test.
• The test follows repo conventions: lives in test/internal/ alongside other build-*.test.ts files, uses tempDir from harness, binds a local net.createServer to 127.0.0.1:0 (no external network), injects a zero-backoff policy so the full 10-attempt loop runs in milliseconds rather than sleeping, and cleans up via await using and mockRestore() in finally. The schedule-timing test is pure arithmetic, not wall-clock.
• No CODEOWNERS entries for scripts/build/. No prior human review comments to address.