Skip to content

fs.promises.watch: implement as a real async generator - #34718

Open
robobun wants to merge 8 commits into
mainfrom
claude/e3fb6844/fs-promises-watch-async-generator
Open

robobun wants to merge 8 commits into
mainfrom
claude/e3fb6844/fs-promises-watch-async-generator

Conversation

@robobun

@robobun robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • fs.promises.watch() opens its native watcher inside the call (src/js/node/fs.promises.ts:100 on main). A watcher that nothing iterates keeps the process alive.
  • The hand-written iterator has one resolver slot (fs.promises.ts:153). A second pending next() replaces it, so the first never settles. throw() and Symbol.asyncDispose are missing.
  • No user reported this. A comparison with Node found it.

Fix

  • watch() is an async function*, as in Node. The body runs on the first next(), pending calls queue, and a finally closes the watcher.
  • Verified: test/js/node/watch/fs.watch.test.ts (15 new tests, the released bun fails 13), test/js/node/watch/, nine Node tests.
  • Self-reviewed: 13 concerns raised, 9 for this PR, 8 addressed. Rejected: a test for the pre-aborted check, which passes with the check deleted.

Background

  • An async generator runs no code before its first next(). It serves one request at a time, oldest first. A return() is a request too.
  • Considered a resolver queue and a lazy start in the hand-written iterator. The generator gives both in 42 fewer lines.

Downsides

  • Breaking, as on Node: pending calls settle oldest first. A loop that abandons next() on a timeout and calls it again sees 0 of 5 events (main: 5 of 5).
  • Breaking, as on Node: errors move to the first next(), so a try around only the watch() call misses ENOENT. return() waits behind a pending next() with the watcher open. A signal ends it.
  • Against main, a delivered event costs +660 instructions (+4.9%) on release builds.
Notes

Breaking changes, as on Node v26.3.0

Linux x64. The Bun columns are one debug build of main 179664e with only the bundled node/fs.promises.js exchanged.

What the program does main this PR Node What to change
Two next() calls are pending, two events come the first never settles both settle both settle
A loop abandons next() on a timeout and calls next() again, 5 events sees 5 of 5 0 of 5 0 of 5 Keep the pending promise: pending ??= it.next()
try { w = watch(missing) } catch {}, with the loop outside the try ENOENT from the call the first next() rejects the first next() rejects Put the loop in the try
Events between watch() and the first next() delivered lost lost Call next() first
return() while a next() is pending settles at once, watcher closed waits for the next event, watcher open waits, watcher open Pass a signal and abort
inotify watches left after 200 x (next(), return()) 0 200 200
Bytes that each abandoned next() keeps (100,000 calls) 1 128 240
Readable.from(watch(dir)).destroy() while the stream waits: 'close' comes at once after the next event after the next event Pass a signal to watch()
return() behind a pending next(), remove and re-create the path, watch it again: the new watcher gets events 10 of 10 rounds 0 of 10 10 of 10 none in JS, see below
1,000 watchers that nothing iterates: inotify watches after a full GC 1,000 0 0
  • The row before the last is a fault of the native watcher, not of the generator. A new watch joins an open watcher of the same path (src/runtime/node/path_watcher.rs), and here the inotify descriptor of that watcher is gone. Plain fs.watch has the fault on main: with a first fs.watch left open, the second watcher got events in 3 of 8 rounds (Node: 8 of 8). fs.watch: watch a macOS file with kqueue, join only a watcher of the same inode #44067 changes the join. This PR moves the fault: it adds the return() case, and it removes the case of a watch() that nothing iterates (main 3 of 8, this PR 8 of 8). I did not run these on macOS or Windows.
  • A published package does the try around the call only. @stacksjs/bun-router@0.1.24 (dist/cli.js) wraps watch(dir, { recursive: true }) in try/catch to skip a missing directory, and runs the loop in an async function that nothing awaits. On main the missing directory is skipped. With this PR, and on Node, the loop rejects with ENOENT and nothing handles it. I read the published file. I did not run the package.

Question for a maintainer

What this PR does not change

  • After an abort, next() rejects at once and drops the events that are still in the queue, as on main. Node yields them first, up to its maxQueue of 2,048.
  • I built Node's order on this branch and took it out again. With it, the next next() after an abort gave one more event in 11 of 20 rounds of one writeFileSync (main and Node: 0 of 20). The native watcher gives a whole batch of events to the listener in one task, so the queue holds events that Node does not have yet. fs.watch: run nextTicks and microtasks between the events of one batch #43715 changes that delivery.
  • The test for await: a body that aborts on an event is not run again keeps the order of main.
  • Node's options maxQueue and overflow are not implemented (fs.promises.watch: implement maxQueue and overflow options #35878 is closed).

Related open PRs (checked with git merge-tree against this head)

Repros

// One resolver slot.
import { promises as fsp } from "node:fs";
import * as fs from "node:fs";
const dir = fs.mkdtempSync("/tmp/watch-");
const it = fsp.watch(dir)[Symbol.asyncIterator]();
const p1 = it.next(), p2 = it.next();
fs.writeFileSync(dir + "/a", "1");
fs.writeFileSync(dir + "/b", "2");
await Promise.all([p1, p2]);          // main: p1 never settles
console.log(typeof it.throw);         // main: "undefined"
// Eager start.
require("fs/promises").watch("/tmp");                 // main: the process never exits
const w = require("fs/promises").watch("/missing");   // main: ENOENT from this call. This PR and Node: from the first next()

try { for await (const e of watch("/missing")) {} } catch {} catches ENOENT on main, with this PR, and on Node. An earlier text of this body said that main misses it. That was wrong.

// The timeout-retry loop.
const it = fsp.watch(dir)[Symbol.asyncIterator]();
while (true) {
  const r = await Promise.race([it.next(), timeout(100)]);
  if (r === TIMEOUT) continue;        // abandons the pending next(), then calls next() again
  seen.push(r.value.filename);
}
// meanwhile: 5 x fs.mkdirSync(dir + "/dN"), 300 ms apart (one event each)
  • On main this loop does not deadlock. The newest next() replaces the resolver slot, so the live call gets each event. The abandoned calls never settle.
  • With oldest-first settlement the abandoned calls get the events, so the live call starves. Every design that settles the first waiter first has this effect, a resolver queue in the hand-written iterator too.

Other differences from main that follow from the generator

  • return() returns a promise. On main it returns a plain { value, done } object.
  • The object is its own iterator (it[Symbol.asyncIterator]() === it), and it has next(). On main watch(dir).next is undefined.
  • await using w = fsp.watch(dir) works. On main it throws TypeError: @@asyncDispose and @@dispose must not be undefined or null and leaks the watcher.
  • The abort listener is added at the first next(). On main it is added in the call.

Tests

  • 15 new tests in the fs.promises.watch block. The released bun (1.4.3-canary.1) fails 13 of them.
  • The other 2 pass there, and that is intended:
    • for await: a body that aborts on an event is not run again keeps the abort order of main. It fails when the order is changed (20 rounds, one rename per step).
    • a watcher error rejects next(), and then the process can exit is a fixture from the root-only fs.watch test. It now also runs when the tests do not run as root.
  • Each of these changes makes a test fail: no removeEventListener in the finally, argument errors thrown from the call, no Symbol.asyncDispose, the outer { [Symbol.asyncIterator]() } object of main, no close in the finally.
  • Not done from the self-review: a test for if (signal?.aborted) throw before the watcher opens. With the line deleted, a pre-aborted signal and a missing path still give the AbortError, so that test cannot fail.

Numbers

  • Instructions: a release build of this generator merged onto main 6d504dd, against that main, linux x64, one core, 15 rounds, spread 0.05% to 0.54%. Open + next() + abort: 37,820 to 39,925 (+5.6%). One delivered event: +660 (+4.9%). Open + one event + return(): +670 (+2.4%). Cycles and wall time are equal. The generator code did not change after that measurement.
  • JS cells, counted today on the debug build with the GC off: per started iterator 28 to 22 with a signal and 27 to 21 without. Per delivered event 14 to 11. The load of node:fs/promises is 1,330 to 1,331 cells.
  • Reads of signal.aborted per delivered event: 2 before and after (Node: 1).
  • The numbers were not taken on macOS or Windows.

Keeping the wake-up from return()

A plain async generator cannot wake a pending next() from return(). An own return property on the generator object can: it closes the watcher, which wakes the body, and then it calls the prototype return. The cost is one closure and one property per watcher. This PR does not do it, because it departs from Node.


no test proof · iteration 10 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/watch/fs.watch.test.ts

@robobun

robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:39 PM PT - Oct 24th, 2026

✅ @robobun, your commit 1164ab606a216f1f568fc9c2166a8ede8b67fae6 passed in Build #132401! 🎉


🧪   To try this PR locally:

bunx bun-pr 34718

That installs a local version of the PR into your bun-34718 executable, so you can run:

bun-34718 --bun

@coderabbitai

coderabbitai Bot commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

fs.promises.watch now uses an async generator that queues filesystem events, supports concurrent consumers, throws on abort, and centralizes watcher cleanup. Tests cover iterator lifecycle, concurrent next() calls, abort behavior, and unconsumed iterator process lifetime.

Changes

Async watcher behavior

Layer / File(s) Summary
Generator event delivery and cleanup
src/js/node/fs.promises.ts
watch uses an async generator with FIFO event queuing, concurrent-consumer wakeups, abort errors, explicit abort-listener removal, and watcher closure in finally.
Iterator lifecycle and concurrency validation
test/js/node/watch/fs.watch.test.ts
Tests cover return() and throw(), concurrent next() calls, abort after an event, and creating an unconsumed iterator without keeping the process alive.

Possibly related PRs

  • oven-sh/bun#34505: Updates underlying fs.watch and FSWatcher lifecycle and native event/error routing used by fs.promises.watch.
Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely states the main change: implementing fs.promises.watch as a real async generator.
Description check Passed The description provides a detailed explanation of the problem, the fix, behavior changes, trade-offs, test coverage, and verification results. It does not use the template headings exactly, but it co…
Full details: Description check

Explanation

The description provides a detailed explanation of the problem, the fix, behavior changes, trade-offs, test coverage, and verification results. It does not use the template headings exactly, but it contains the required information under equivalent sections.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/js/node/fs.promises.ts`:
- Around line 117-124: Remove the unreachable "close" and "error" branches from
the event-processing loop consuming watcher events, leaving it to yield each
filesystem change event. Apply the root-cause listener wiring fix on the watcher
separately as requested by the related comment, rather than preserving dead
checks in this loop.

In `@test/js/node/watch/fs.watch.test.ts`:
- Around line 908-918: Extend the async iterator protocol test around it.throw
to invoke it.throw with a representative error and assert the resulting Promise
follows the expected rejection or closed-generator semantics, while preserving
the existing return() and next() assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3aa195cf-2f0a-4ebc-950c-b4f0b9c4aa23

📥 Commits

Reviewing files that changed from the base of the PR and between 98f6649 and 2bac0c6.

📒 Files selected for processing (2)
  • src/js/node/fs.promises.ts
  • test/js/node/watch/fs.watch.test.ts

Comment thread src/js/node/fs.promises.ts Resolved
Comment thread test/js/node/watch/fs.watch.test.ts Resolved
Comment thread src/js/node/fs.promises.ts Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
test/js/node/watch/fs.watch.test.ts (1)

947-949: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Do not discard iterator cleanup failures.

Both cleanup paths catch every return() rejection, so tests can pass while iterator shutdown or finally cleanup is broken. Assert the expected closed result, or the specific expected abort error, instead of using catch(() => {}).

As per coding guidelines, tests must not silently weaken a safety net, and assertions should verify the strongest invariant.

Also applies to: 965-967

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/js/node/watch/fs.watch.test.ts` around lines 947 - 949, Update the
iterator cleanup in the finally blocks around ac.abort() and it.return() so
return() rejections are not swallowed. Assert the expected closed iterator
result or the specific expected abort error, preserving validation of shutdown
and finally cleanup behavior in both affected cleanup paths.

Source: Coding guidelines

src/js/node/fs.promises.ts (1)

81-87: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Make watch() shutdown wake a blocked idle iterator. return()/throw() can stay pending while the generator sits at await promise, so finally never runs and the watcher can keep the native handle alive. Add a regression that calls return()/throw() after priming the iterator with next() and before any event arrives.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/js/node/fs.promises.ts` around lines 81 - 87, The watch iterator’s
shutdown path must wake the promise awaited by the idle generator so return() or
throw() can complete and run cleanup. Update the pendingResolve/wake mechanism
in watch() to settle the blocked await during shutdown, while preserving normal
event delivery, and add regression coverage that primes next() before invoking
both return() and throw() without emitting an event.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/js/node/fs.promises.ts`:
- Around line 81-87: The watch iterator’s shutdown path must wake the promise
awaited by the idle generator so return() or throw() can complete and run
cleanup. Update the pendingResolve/wake mechanism in watch() to settle the
blocked await during shutdown, while preserving normal event delivery, and add
regression coverage that primes next() before invoking both return() and throw()
without emitting an event.

In `@test/js/node/watch/fs.watch.test.ts`:
- Around line 947-949: Update the iterator cleanup in the finally blocks around
ac.abort() and it.return() so return() rejections are not swallowed. Assert the
expected closed iterator result or the specific expected abort error, preserving
validation of shutdown and finally cleanup behavior in both affected cleanup
paths.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f4ddd6c1-e39e-4f22-955b-9834b09358e2

📥 Commits

Reviewing files that changed from the base of the PR and between 9e99954 and 23e3ec7.

📒 Files selected for processing (2)
  • src/js/node/fs.promises.ts
  • test/js/node/watch/fs.watch.test.ts

@robobun

robobun commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator Author

Re: the two out-of-diff review findings:

it.return().catch(() => {}) in test cleanup (test:947-949, 965-967): removed in c688c43. The concurrent-next cleanup now awaits return() directly; the abort-between-yields test asserts return() resolves {value: undefined, done: true} on a completed generator.

return()/throw() pending while parked at await (src:81-87): this matches Node and the async generator spec. AsyncGenerator.prototype.return queues behind an in-flight await; the request is processed only when the await settles. Verified against Node:

const it = fsp.watch(dir)[Symbol.asyncIterator]();
it.next();            // parks at await (no events)
await it.return();    // hangs on Node too

Node's own fsPromises.watch is an async function* with the same shape and exhibits the same behavior. Making return() wake the blocked await would diverge from Node, so leaving as-is. The practical escape hatch is the one Node documents: pass an AbortSignal and abort it, which does wake the await (via onAbort -> wake()).

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The abort-between-yields regression I flagged earlier is fixed (23e3ec7) and covered by a new test; the current diff looks correct to me. Deferring to a human because this is a full rewrite of the fs.promises.watch iterator with observable Node-compat behavior shifts (native handle now opens on first next(), validation/ENOENT surface async instead of sync, return() no longer wakes a parked next()) that a maintainer should ratify.

What was reviewed:

  • Traced the flattened loop: signal.aborted is now re-checked after every yield and every await, so abort-during-yield throws AbortError instead of resolving {done:true} via the close sentinel.
  • finally runs on return/throw/error and idempotently closes the watcher + detaches the abort listener; concurrent next() is serialized by the async-generator queue so a single pendingResolve slot is sufficient.
  • Confirmed fs.promises.watch has no other test-suite consumers that depend on the old synchronous-throw timing.
Extended reasoning...

Overview

Rewrites watch() in src/js/node/fs.promises.ts from a hand-rolled {[Symbol.asyncIterator]() { return {next, return} }} object into a real async function*, matching Node's implementation. Adds four tests to test/js/node/watch/fs.watch.test.ts covering iterator shape (return()/throw() return Promises), concurrent next(), abort-while-suspended-at-yield, and never-iterated process exit.

Security risks

None. Pure JS iterator-protocol reshaping over the existing native watcher binding; no new I/O, parsing, or trust boundaries.

Level of scrutiny

Medium. Built-in node:fs/promises module code with subtle async-generator control flow. The first revision had a real regression (my earlier inline finding: aborting between yields resolved {done:true} instead of throwing AbortError because the close sentinel was drained before the abort re-check). That was fixed by flattening the loop to one queue item per iteration so the abort check sits between every yield/await, with a regression test added. I re-traced the fixed flow and it holds.

Other factors

  • All CodeRabbit threads and my prior thread are resolved; the bug-hunting pass on the latest commits found nothing.
  • Grep confirms test/js/node/watch/fs.watch.test.ts is the only test file exercising fs.promises.watch, so the sync→async error-timing shift shouldn't silently break other suites.
  • The intentional behavior changes (lazy native-handle open, ENOENT/validation on first next(), return() queuing behind an in-flight internal await per spec/Node) are defensible Node-compat moves but are user-visible enough that a maintainer should sign off rather than auto-approving.
  • Test cleanup paths look sound: after Promise.all([p1,p2]) the generator is at suspendedYield, so ac.abort(); await it.return() completes without swallowed rejections.

@robobun

robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status. Head 1164ab6, on main 179664e. The PR waits for a maintainer's yes or no (see the question in the Notes of the body). CI runs on this head.

How I reproduced the problem (linux x64, Node v26.3.0 as the reference):

  • One resolver slot: two pending next() calls, then two events. On main the first call never settles. With this PR and on Node both settle.
  • Eager start: require("fs/promises").watch("/tmp") with nothing else. On main the process never exits. With this PR and on Node it exits.
  • Shape: on main watch(dir) has no next(), throw() or Symbol.asyncDispose, and return() does not return a promise.

How I checked the fix.

  • bun bd test test/js/node/watch/fs.watch.test.ts: 15 new tests. With the src/ of main, 13 of them fail. With this PR all pass. The tests that wait for file events do not depend on when the watcher starts or on how many events one write gives.
  • test/js/node/watch/ (80 pass, 0 fail), nine Node tests for fs.promises.watch and fs.watch, and the five fsPromisesWatch tests of module-graph-isolation.test.ts.
  • tsc -p src/js/tsconfig.json.

I did not run anything on macOS or Windows.

@robobun
robobun force-pushed the claude/e3fb6844/fs-promises-watch-async-generator branch from c688c43 to 7e37e67 Compare July 21, 2026 08:55

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found — the abort-between-yields regression I flagged earlier is fixed (abort re-checked at the top of every loop iteration) and covered by a test. Deferring to a maintainer since this shifts observable Node-compat timing (validation/ENOENT now surface on first next() instead of synchronously).

What was reviewed:

  • Generator control flow: finally closes the watcher and detaches the abort listener on every exit path (return/throw/return()/throw()); closeWatcher() is idempotent.
  • Single pendingResolve slot is safe because async-generator machinery serializes body execution — only one await can be pending.
  • return()-while-parked-at-await hanging matches Node/spec (author verified); close/error sentinel branches are reachable via the internal binding's multiplexed callback.
  • New tests: concurrent next(), return()/throw() shape, abort-at-yield, never-iterated exit, ENOENT-on-first-iteration — all wire failures to reject and clean up in finally.
Extended reasoning...

Overview

Rewrites fs.promises.watch in src/js/node/fs.promises.ts from a hand-rolled {next, return} iterator into a real async function*, matching Node's implementation shape. The generator body opens the native watcher lazily on the first next(), queues events via $createFIFO, re-checks signal?.aborted between every yield/await, and closes the watcher + detaches the abort listener in finally. Adds five tests to test/js/node/watch/fs.watch.test.ts.

Security risks

None. This is iterator-protocol plumbing over an existing native watcher binding; no new input parsing, no auth/crypto, no untrusted data handling beyond what already existed.

Level of scrutiny

Moderate. It's a self-contained rewrite of one function, but it's user-facing Node-compat surface with intentional observable-behavior changes: argument validation and ENOENT now reject the first iteration instead of throwing synchronously from watch(), and an un-iterated watch() no longer pins the event loop. Both match Node, but a maintainer should confirm they want that timing shift (it could break code that relied on the old synchronous throw, even though that reliance was itself a Node-incompat).

Other factors

  • My prior inline finding (abort while suspended at yield completing {done:true} instead of throwing AbortError) was fixed by flattening to one queue item per outer iteration and adding a regression test.
  • CodeRabbit's two comments (dead-branch false positive, it.throw() coverage) were both resolved; robobun's rebuttal on the internal-binding callback contract is correct — I checked src/js/internal/fs/watch.ts uses the same "close"/"error" eventType dispatch.
  • The author verified return() pending behind an in-flight await matches Node and the async-generator spec, so that's not a regression.
  • Test coverage is thorough for the specific protocol fixes; existing fs.promises.watch tests in the file continue to exercise the for-await path. Prior CI run had this test file green on all lanes.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun rebase

@robobun
robobun force-pushed the claude/e3fb6844/fs-promises-watch-async-generator branch from 7e37e67 to bafe978 Compare August 4, 2026 07:46
@robobun

robobun commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (83c7055). One conflict in src/js/node/fs.promises.ts: main had added a watcher.close() call in the old iterator's error branch, which is subsumed by the finally { closeWatcher() } here. All 48 fs.watch.test.ts tests pass locally.

Comment thread src/js/node/fs.promises.ts Resolved
Comment thread src/js/node/fs.promises.ts Resolved
Comment thread src/js/node/fs.promises.ts Resolved
Comment thread src/js/node/fs.promises.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both earlier findings (abort-between-yields regression, comment-cop flags) are addressed, and the current revision has no new issues.

What was reviewed

  • Generator control flow: abort re-checked between every yield/await; finally closes the watcher and detaches the abort listener on all exits (return/throw/close/error/ENOENT).
  • Pre-aborted signal and ENOENT paths throw before the native handle exists, so nothing leaks; never-iterated watch never opens a handle.
  • Test cleanup paths: it.return() after abort on a suspended generator, and the abort-between-yields test's watcher lifetime on assertion failure — ruled out as a leak concern.
  • CI: fs.watch.test.ts green on all lanes; the one failure (worker-transfer-terminate-stress) is a documented unrelated flake.
Extended reasoning...

Overview

This PR rewrites fs.promises.watch() in src/js/node/fs.promises.ts from a hand-rolled {next, return} iterator into a real async function*, matching Node's implementation shape. Net ~50 lines removed. Five new tests in test/js/node/watch/fs.watch.test.ts cover concurrent next(), return()/throw() shape, abort-while-suspended-at-yield, never-iterated process exit, and ENOENT surfacing on first iteration.

Security risks

None. This is a JS-side async iterator wrapper around the existing native watcher binding. No new syscalls, no path handling changes, no untrusted input parsing beyond what the old code already did (validateAbortSignal, filename coercion). The __proto__: null on yielded events is preserved.

Level of scrutiny

Medium. It's a Node-compat behavioral change (errors now surface on first next() instead of synchronously), but that's the documented intent and matches Node exactly. The function is self-contained; no other callers in the tree depend on the old synchronous-throw shape. The generator's finally block correctly handles every exit path I traced: normal return on 'close', throw on 'error', it.return()/it.throw() from the consumer, ENOENT before the try, and pre-aborted signal before the watcher exists.

Other factors

  • I reviewed this PR twice previously. The first pass found a real regression (abort while suspended at yield returned {done:true} instead of throwing AbortError) — fixed with a flat one-item-per-iteration loop and a dedicated regression test. The second pass flagged comment-cop violations — trimmed. Both threads resolved.
  • CodeRabbit's two findings (dead-code branches, throw() not exercised) were both addressed/refuted; the 'close'/'error' branches are reachable via the internal binding's multiplexed callback contract.
  • The bug-hunting system found nothing this run; one candidate (test cleanup leaking a watcher on assertion failure) was verified and ruled out.
  • Promise.withResolvers() (non-primordial) matches the pre-existing pattern in this file and the removed code.
  • CI build 88755: fs.watch.test.ts passes on all lanes; the sole failure is the known-intermittent worker-transfer-terminate-stress SIGABRT on x64-asan, unrelated to fs.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found — the implementation looks correct and all prior review threads are resolved. Deferring only because this is a public-API rewrite with an observable timing change (ENOENT / arg-validation errors now surface on the first next() instead of synchronously from watch()), which a maintainer should sign off on.

What was reviewed:

  • Generator control flow: abort re-checked between every yield/await; finally closes the watcher and detaches the abort listener on all exits (return, throw, error, close sentinel).
  • Single pendingResolve slot is safe because the async-generator queue serializes body execution — only one await promise is ever live.
  • eventType === "close"/"error" branches are reachable via the internal binding's multiplexed callback (same contract as internal/fs/watch.ts #onEvent).
  • Test cleanup paths: ac.abort() before await it.return() wakes any parked await so return() can't hang behind a queued next().
Extended reasoning...

Overview

Rewrites fs.promises.watch (src/js/node/fs.promises.ts) from a hand-rolled {next, return} iterator to a real async function*, matching Node's implementation shape. The generator body opens the native watcher on first next(), drains a FIFO of {eventType, filename} events, re-checks signal.aborted between every yield, and closes the watcher / detaches the abort listener in a finally. Five new tests in test/js/node/watch/fs.watch.test.ts cover concurrent next(), return()/throw() shape, abort-while-suspended-at-yield, never-iterated process exit, and ENOENT-on-first-iteration.

Security risks

None. This is a JS-side control-flow refactor of an existing Node-compat API; no new attack surface, no untrusted-input parsing, no privilege boundaries touched.

Level of scrutiny

Medium-high. The function itself is small (~50 lines), but async-generator + abort + native-close-sentinel interaction is subtle — my earlier review of this PR caught a real regression (abort between yields resolving {done:true} instead of throwing AbortError), which was fixed with a regression test. The current revision has been through CodeRabbit, comment-cop, and two rounds of my inline findings; all threads are resolved and the bug hunter found nothing this run.

Other factors

  • Observable behavior change: because the generator body runs on first next(), ENOENT, ERR_INVALID_ARG_TYPE, and pre-aborted-signal errors now reject the first iteration rather than throwing synchronously from watch(). This matches Node exactly and is documented in the PR description, but it's a user-visible semantics shift for anyone wrapping fs.promises.watch() in a synchronous try/catch. That's the one thing I'd want a maintainer to explicitly acknowledge before merge.
  • Node parity on return() while parked at await: the author verified (and documented in a PR comment) that return() queuing behind an in-flight await matches both the async-generator spec and Node's own fsPromises.watch; the escape hatch is AbortSignal, which does wake the await via onAbort → wake().
  • CI: build 88755 (pre-rebase head) passed fs.watch.test.ts on all lanes; the post-rebase build 88857 was retriggered at the current head.
  • Test quality: new tests follow harness conventions (bunEnv/bunExe, await using for spawns, drain stdout/stderr/exited concurrently, repeat() polling instead of sleeps, abort-signal cleanup in finally).

@robobun

robobun commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

This change also fixes await using on the value that fs.promises.watch() returns.

On main, watch() returns a plain object with only [Symbol.asyncIterator]. The object has no Symbol.asyncDispose. So await using w = fsp.watch(dir) throws TypeError: @@asyncDispose and @@dispose must not be undefined or null. Because watch() opens the native watcher before the first next(), the watcher from the initializer also leaks and keeps the process alive. Node returns an async generator object. It inherits Symbol.asyncDispose from %AsyncIteratorPrototype%, so the same code works there.

Repro:

import fsp from "node:fs/promises";
const w = fsp.watch(".");
console.log("asyncDispose:", typeof w[Symbol.asyncDispose]);
try { { await using x = w; } console.log("ok"); } catch (e) { console.log("THREW", e.message); }
const t = setTimeout(() => { console.log("loop still alive after 1s (watcher leaked)"); process.exit(0); }, 1000); t.unref();

Bun 1.4.0 and main print:

asyncDispose: undefined
THREW @@asyncDispose and @@dispose must not be undefined or null
loop still alive after 1s (watcher leaked)

Node v26.3.0 prints asyncDispose: function, ok, and exits at once. A debug build of main with the src/js/node/fs.promises.ts change from this branch prints the same as Node. I also checked a started generator: after one next() resolves with an event, the end of the await using block closes the watcher and the process exits. Node v26.3.0 does the same.

A test for this shape would lock it in. For example: typeof fsp.watch(dir)[Symbol.asyncDispose] is "function", and a child process that leaves an await using block after one event exits on its own.

@robobun

robobun commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

I ran the consumer that the original report names, and the result changes the merge decision for this PR.

The report says that a Promise.race([it.next(), timeout]) retry loop deadlocks on main. It does not. On main the newest next() replaces the resolver slot, so the live call gets each event. Only the abandoned calls never settle.

With this PR, pending calls settle oldest first, as in Node. Each event goes to an abandoned call, and the live call starves.

runtime events the loop saw (5 events, 3 runs each)
main (1.4.3-canary.1+367d939d9) 5 of 5
this branch merged onto 367d939 (debug build) 0 of 5
Node v26.3.0 0 of 5
const it = fsp.watch(dir)[Symbol.asyncIterator]();
while (true) {
  const r = await Promise.race([it.next(), timeout(100)]);
  if (r === TIMEOUT) continue; // abandons the pending next(), then calls next() again
  seen.push(r.value.filename);
}
// meanwhile: 5 x fs.mkdirSync(dir + "/dN"), 300 ms apart (one event each)

This is the other side of the fix, not of the generator. Every design that settles the first waiter first has it, a resolver queue in the hand-written iterator too. A loop that keeps its pending promise across timeouts (pending ??= it.next(), race pending, clear it when it settles) sees 5 of 5 on main, on this branch, and on Node.

The PR still fixes Promise.all([it.next(), it.next()]), the return() and throw() shape, Symbol.asyncDispose, the watcher that nothing iterates, and the ENOENT timing. But it makes a loop that works on main today lose every event, with no error. That is Node's behavior. Whether Node parity is worth that here is a maintainer's decision. The Downsides section and the Notes in the PR body now have these numbers. I did not change the code.

@robobun

robobun commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator Author

#46218 changes the abort path of the fs.promises.watch() iterator that this PR replaces. The two PRs conflict in src/js/node/fs.promises.ts.

What #46218 does:

  • The abort listener wakes next(). watch() adds it at the first next(), with resistStopPropagation, as Node does. On main the callback of the native watcher wakes next() before every abort listener.
  • An aborted next() settles one tick later, as the yield* of Node's fsPromises.watch() does (on every platform since Node v26.4.0).
  • Off macOS and Windows, a next() that waits on a recursive watcher gets no extra tick, as with Node's JS watcher for recursive.

The order of the two PRs:

A maintainer decides which lands first.

The hand-rolled {next, return} iterator had a single nextEventResolve
slot, so a second pending next() overwrote the first's resolver and the
first waiter hung forever. return() returned a plain object instead of a
Promise, and throw() was missing entirely.

Replace it with an async function* like Node does. The generator
machinery queues concurrent next() calls by construction and provides
spec-compliant return()/throw() from %AsyncGeneratorPrototype%.

The native handle is now opened on the first next() (inside the
generator body, like Node), so an fsp.watch() that is never iterated no
longer leaks a watcher that keeps the process alive, and ENOENT is
surfaced on the first next() rather than synchronously.
Resuming from yield landed inside the inner drain loop, skipping the
signal?.aborted check. An abort that fired while suspended at yield would
drain buffered events (including the close sentinel pushed by the native
close) and resolve done:true instead of throwing AbortError.

Flatten to one queue item per outer iteration so the abort check runs
between every yield/await.
…e abort order

- return(), throw(), Symbol.asyncDispose and break each end the iterator,
  remove its abort listener and close the watcher.
- return() waits behind the pending next() calls.
- Argument errors reject the first next(), not the call.
- A for-await body that aborts on an event is not run again. This replaces
  the one-write form of the abort test.
- The watcher-error fixture moves out of the root-only test, so that it
  also runs when the tests do not run as root.
@robobun
robobun force-pushed the claude/e3fb6844/fs-promises-watch-async-generator branch from 2b543e8 to 2ebaef6 Compare October 25, 2026 04:16
@robobun

robobun commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

This PR needs a yes or no from a maintainer before a merge. The head is now 2ebaef6, on main 179664e.

What the PR is. fs.promises.watch() as an async function*, as in Node. Nothing more. The generator code did not change in this push.

What this push adds.

  • 9 more tests: each of return(), throw(), Symbol.asyncDispose and break ends the iterator, removes its abort listener and closes the watcher. Argument errors reject the first next(). return() waits behind the pending next() calls.
  • The abort test is now a 20-round form with one rename per step.
  • The watcher-error fixture moved out of the root-only test, so that it also runs when the tests do not run as root.
  • The breaking label, and a body that lists every break.

What I found and corrected.

  • The body said that a try around the for await misses ENOENT on main. That was wrong. Only a watch() call outside the try differs.
  • The generator breaks more programs than the body said. The Notes of the body now have the whole table, each row with main, this PR and Node v26.3.0.
    • A loop that abandons next() on a timeout and calls it again: 5 of 5 events on main, 0 of 5 here and on Node.
    • A try around only the watch() call no longer catches ENOENT. @stacksjs/bun-router@0.1.24 does this to skip a missing directory (read in its dist/cli.js, not run).
    • return() behind a pending next() waits for the next event, and the watcher stays open. On Linux a new watch of that path, after the path is removed and created again, then got events in 0 of 10 rounds (main and Node: 10 of 10). The cause is the join in the native watcher, which fs.watch: watch a macOS file with kqueue, join only a watcher of the same inode #44067 changes.

What is not in this PR. A report said that after an abort the iterator drops the events that are still in its queue, and that Node yields them first. That is true, on main too. I built Node's order and took it out again: with it, the next next() after an abort gave one more event in 11 of 20 rounds of one writeFileSync (main and Node: 0 of 20), because the native watcher gives a whole batch to the listener in one task. That change needs #43715 first.

The question. Is Node's behaviour worth the breaks in the table? No user reported the problem that this PR fixes.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commits address the earlier abort-at-yield thread: the loop now re-checks signal?.aborted at the top of every iteration after continue, so an abort that fires while the generator is suspended at yield throws AbortError instead of completing on the close sentinel. Beyond the inline findings, I also checked that the finally's closeWatcher() after a native close event is safe (native close() in src/runtime/node/node_fs_watcher.rs is guarded by a closed flag), that the single pendingResolve slot is sufficient now that only one generator body runs at a time, and that queue is a $createFIFO, so shift() is not the Array#shift pattern.

Extended reasoning...

The change rewrites fs.promises.watch in src/js/node/fs.promises.ts into a native async generator with lazy validation and lazy watcher creation, a single finally for all exit paths, and adds a suite of iterator-protocol and exit-path tests in test/js/node/watch/fs.watch.test.ts. No security-sensitive surface is touched. The prior-run inline finding (abort while suspended at yield) was fixed by commit 4adb1a2, and the comment-cop flags were addressed. The inline findings posted this run concern macOS timing in the new spawned fixture and the queued-return() ordering test, and are the reason a human should still look before merging.

Comment thread test/js/node/watch/fs.watch.test.ts Resolved
Comment thread test/js/node/watch/fs.watch.test.ts Resolved
…on events per write

- The fixture for break, return(), throw() and await using writes until each
  exit is done. It waited for a second event from one write before.
- The return() test compares the order of the calls, not the file names.
- The rounds of the abort test run at the same time.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

@robobun

robobun commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

#46937 is stacked on this branch. It adds maxQueue and overflow to this generator in one commit (+40/-3 in src/).

One more row for the question above. Node.js also bounds the queue when neither option is passed: it keeps 2,048 unread events and drops each later one with a warning. #46937 leaves that case unbounded, as main does, until a maintainer decides.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants