Skip to content

fs.watch: deliver every record the OS reports on POSIX, as libuv does - #46191

Open
robobun wants to merge 1 commit into
mainfrom
robobun/ae61af86/fs-watch-deliver-every-record
Open

robobun wants to merge 1 commit into
mainfrom
robobun/ae61af86/fs-watch-deliver-every-record

Conversation

@robobun

@robobun robobun commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #45954

Problem

  • fs.watch(file) on Linux drops the change event of the last write in a burst. A listener that stats the file on each event never sees the final size.
  • ChangeEvent::should_emit (src/runtime/node/path_watcher.rs:226) discards a record of the same path and kind within 1 ms of the last emitted one. Nothing re-arms a trailing event.

Fix

Background

  • PathWatcher is one OS watch. Its reader thread calls emit once per kernel record.
  • Considered a reset of the memo at each flush: it still folds a write and a chmod of one read where node gives two events. Deletion also removes a clock read and a hash per record.

Downsides

Notes

Measurements. Linux x64, release builds: main is 1.4.3-canary+367d939d9 (the watcher code did not change between it and the merge base), this PR is 1.4.4-canary+15ff22a27 (before #44007 was folded in, which does not touch the file path), node v26.3.0. The writer child is bun 1.4.3 in every run. The listener spins 20 us per call.

Workload main: calls / max drift of a 5 ms timer / max RSS This PR Node
200000 appends, one file 158-255 / 7-42 ms / 38-40 MB 251-51219 / 5-1195 ms / 42-66 MB 2889-6995 / 11-58 ms / 68-71 MB
100000 append+chmod pairs 116-239 / 5-79 ms / 38-40 MB 173412-200000 / 3.9-6.7 s / 64-87 MB 23361-35595 / 0.59-1.02 s / 69-72 MB
2000 appends, 50 us apart 150-346 / 4-39 ms / 38-40 MB 966-1393 / 6-76 ms / 42-43 MB 1102-1845 / 3-46 ms / 69-71 MB
  • Trailing write missed (200 appends, 0.3 ms pause, one more append, 20 trials): main 5/20, this PR 0/20, node 0/20.
  • The reporter's run.sh: this PR reached DONE in 5 of 5 runs.
  • Recursive watch, mkdir n then a file in it, 100 runs: n/f reported 1.00 times on main, this PR and node.
  • Node reads inotify on its loop thread, so its kernel queue fills while JS runs, and the kernel merges identical records and drops past max_queued_events. Bun's reader thread drains at once, so a flood yields more records. A bound on that queue is a separate change.
  • Not measured: binary size and instructions per record. bloaty, perf, valgrind and strace are not on this machine, and no release build of the merge base exists here. The change is a deletion on the reader path: one clock_gettime, one wyhash of the path and 24 bytes per handler per record are gone.

Fail-before detail. On bun 1.4.3, 5 runs of the 7 new tests: the write, chmod, write test fails 5 of 5, and 3 or 4 of the 6 recursive tests fail per run (the 1 ms rule hides the second report only when the two records land in time).

Other PRs. #44008 is superseded: it removed the rule on Linux and FreeBSD but replaced it with a merge into an undelivered event, kept the Windows rule, and is stacked on #44007 and #43715. #44007 is folded in here. #40200 and #42819 keep should_emit and must drop it when they rebase. #32962 and #33110 bypassed the rule for one event each (IN_IGNORED, queue overflow) with the same libuv-parity reason. No maintainer has ruled on the rule itself. The two questions put on #44008 are answered here by deletion on POSIX and by keeping Windows until #39827.

Windows. win_watcher.rs keeps ChangeEvent. Its timestamp is libuv's cached loop time, which changes once per loop iteration, so same-name records of one completion fold into one event, and so do records of the next iteration within 1 ms. The listener can run between two iterations, so a trailing write can still be lost there. Deleting the copy now would raise the removed-directory storm from about 500 calls per second to node's 180000 with no Windows test run here. After #39827 retires the watch of a removed directory, the copy can go.

The recursive walk. An entry created after a new subdirectory's watch attaches is reported by the walk and again by its own IN_CREATE, which the next read(2) returns. The 1 ms rule hid that only when the read landed in time. Node's recursive_watch.js has the same race. The vendored node tests use mustCallAtLeast there.

Self-review. 11 concerns raised, 10 addressed: the recursive double report (folded #44007 in), a burst test that passed on 1.4.3 in most runs and would hang on macOS (replaced by the write, chmod, write test), the Windows storm (kept the Windows copy), stale comments, and this body names the superseded PR and the precedents. Rejected: delete the Windows copy now for parity, for the reason above.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/watch/fs.watch.test.ts

Delete the per-handler 1 ms duplicate suppression in PathWatcher::emit.
It dropped the inotify record that carried the last write of a burst, so
a listener never heard of the final change. The kqueue and FSEvents arms
share the same function.

Under a recursive watch, the reader now skips the nameless IN_ATTRIB,
IN_MODIFY, IN_DELETE_SELF and IN_MOVE_SELF records of a subdirectory's
own watch. The parent's watch reports the same change under the name of
the subdirectory. The suppression hid that second report when the two
records landed within 1 ms.

The Windows copy of the rule stays. It caps the storm of renames that
libuv reports for a removed directory until the watch of a removed
directory is retired.
@robobun

robobun commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:10 AM PT - Oct 20th, 2026

✅ @robobun, your commit 00e4ffcdde319e5abbe77cf8df247b0ab60b441b passed in Build #129505! 🎉


🧪   To try this PR locally:

bunx bun-pr 46191

That installs a local version of the PR into your bun-46191 executable, so you can run:

bun-46191 --bun

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: oven-sh/bun/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: b430c89a-66d0-4eab-89a1-46f9483e01a9
📥 Commits

Reviewing files that changed from the base of the PR and between ea6a2c7 and 00e4ffc.

📒 Files selected for processing: 3
  • src/runtime/node/node_fs_watcher.rs
  • src/runtime/node/path_watcher.rs
  • test/js/node/watch/fs.watch.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Walkthrough

Linux fs.watch dispatch no longer coalesces duplicate events per handler. Recursive handling skips selected nameless events from non-root owners. New Linux tests cover event ordering and recursive directory operations.

Changes

Linux fs.watch Event Delivery

Layer / File(s) Summary
Uniform event dispatch
src/runtime/node/node_fs_watcher.rs, src/runtime/node/path_watcher.rs, test/js/node/watch/fs.watch.test.ts
Handlers no longer maintain duplicate-suppression state, and dispatch forwards each delivered record to every handler. The Linux test checks ordered "change" events followed by "rename".
Recursive watch event handling
src/runtime/node/path_watcher.rs, test/js/node/watch/fs.watch.test.ts
Recursive handling skips specified nameless events from non-root owners. Linux tests cover directory removal and rename operations, modification-time changes, and removal while a file descriptor remains open.

Suggested reviewers: jarred-sumner

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 00e4f

This change makes Linux fs.watch deliver every event the OS reports, which addresses the dropped trailing write events. No actionable merge-blocking risk was identified in the supplied context.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary change: POSIX fs.watch now delivers every OS-reported record, matching libuv behavior.
Linked Issues check Passed Issue #45954 requires Linux fs.watch to deliver a trailing event after an append burst. PathWatcher::emit now delivers every POSIX record and removes the 1 ms suppression. The Linux test adds cove…
Out of Scope Changes check Passed The changes stay within the stated watcher fix. The recursive subdirectory filtering and tests are stated parts of this PR because recursive self-records were folded into the fix. The WatchEventKind…
Description check Passed The description is complete and relevant. It explains the problem, the fix, known tradeoffs, platform-specific behavior, and verification results. It does not use the template headings exactly, but it…
Full details: Linked Issues check

Explanation

Issue #45954 requires Linux fs.watch to deliver a trailing event after an append burst. PathWatcher::emit now delivers every POSIX record and removes the 1 ms suppression. The Linux test adds coverage for multiple same-path records, including the final write record. The reported reproduction also reaches DONE in 5 of 5 runs.

Full details: Out of Scope Changes check

Explanation

The changes stay within the stated watcher fix. The recursive subdirectory filtering and tests are stated parts of this PR because recursive self-records were folded into the fix. The WatchEventKind derive cleanup supports removal of ChangeEvent. No unrelated behavior change is demonstrated.

Full details: Description check

Explanation

The description is complete and relevant. It explains the problem, the fix, known tradeoffs, platform-specific behavior, and verification results. It does not use the template headings exactly, but it provides the required information through the Problem, Fix, and Verified sections.

✨ Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline findings, I also checked two things and ruled them out: the new oi += 1; continue skip at src/runtime/node/path_watcher.rs:1029 bypasses touched.get_or_put, but since nothing was emitted for that owner no on_update_end is owed, and the parent wd's named record touches the same watcher in the common case; and the Darwin on_fs_event path (src/runtime/node/path_watcher.rs:1267) also loses the 1 ms fold, but FSEvents already coalesces per path in the kernel before events_cb, so it does not produce the new same-path duplicates inotify can.

Extended reasoning...

The change deletes the per-handler ChangeEvent duplicate suppression in src/runtime/node/path_watcher.rs so every inotify, kqueue and FSEvents record reaches every FSWatcher handler, adds a Linux-only skip of nameless self-records on a recursive watch's subdirectory wds, and adds Linux-gated tests. It touches no security-sensitive surface. Four confirmed findings are posted inline (listener flood without a bound, mount-root attribute events lost under the new skip, double reports from the recursive walk race, and the pre-existing Windows gap), so the inline comments already signal that a human needs to look; this note only records what else was examined.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread src/runtime/node/path_watcher.rs Resolved
Comment thread src/runtime/node/path_watcher.rs Resolved
Comment thread src/runtime/node/path_watcher.rs Resolved
Comment thread src/runtime/node/path_watcher.rs Resolved
@robobun

robobun commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

#44007 and #44008 worked on the same rule. I closed both: this PR contains the change of #44007, and it removes the rule with no stack. Two facts from that work, measured on this head (00e4ffc, release build, Linux x64, load average 5):

#44008 dropped a record identical to a listener's last event while the listener has not received that event. With that merge the first number was 8 of 2000 in the runs of #44008. The code is here if a maintainer wants that bound after this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fs.watch on Linux delivers only 2-4 events for a burst of 2000 appends from two processes, and the last writes never produce an event

1 participant