Repository navigation
Conversation
… (Linux) The shell writes `$` output to dups of fd 1 and fd 2. For a poll that its owner did not label, the pipe writer waited for POLLOUT and then called write(2) with the whole buffer. POLLOUT on a Linux pipe proves one free slot, and a slot holds one page. write(2) of more than that on a description without O_NONBLOCK returns only when every byte is in, so the event loop thread slept until the reader drained the pipe. The shell cannot set O_NONBLOCK on that description. Children inherit it, and a plain tool fails with EAGAIN on it. pwritev2(RWF_NOWAIT) covers a pipe(2) pipe from Linux 6.4, a socket from 6.5, and a FIFO never. Once RWF_NOWAIT is refused, try_write looks at the fd on its first write of more than one page (Linux and Android): - A pipe or FIFO: poll, then write(2) of at most one slot, while the description has no O_NONBLOCK. #43868 had this bound in bb0e24a and removed it in b2ca90a, when process.stdout went back to O_NONBLOCK. On macOS the byte count from kqueue already bounds this write. - A socket that its owner labelled as a pipe: it becomes a socket, and send(MSG_DONTWAIT) writes it. - A tty: unchanged. IOWriter::bump moved every unwritten byte to the front each time 128 KB were written. That is quadratic in the queue wherever writes do not wait, on every platform. It now waits until the written prefix is at least half of the buffer.
|
Updated 12:36 PM PT - Oct 26th, 2026
❌ @robobun, your commit 584ed1b has 4 failures in
🧪 To try this PR locally: bunx bun-pr 46307That installs a local version of the PR into your bun-46307 --bun |
|
Status Reproduced on Linux 6.1.155 with release builds of main (71bc818) and of this branch.
PR: #46307 |
|
No actionable comments were generated in the recent review. ℹ️ Recent review info
WalkthroughPOSIX pipe writers now track per-descriptor write strategies and select write paths based on descriptor type and flags. Shell output buffer draining also requires the written prefix to cover at least half the buffer. New tests cover FIFO, socket, and TTY output. ChangesShell Output Handling
Suggested reviewers: Priority: Merge Risk:
 |
| Check name | Status | Explanation |
|---|---|---|
| Title check |  |
The title clearly summarizes the primary Linux change: limiting blocking pipe and FIFO writes to one slot per POLLOUT for stdout and stderr. |
| Linked Issues check | |
Check skipped because no linked issues were found for this pull request. |
| Out of Scope Changes check | |
Check skipped because no linked issues were found for this pull request. |
| Description check | |
The PR description is detailed and directly related to the changeset. It describes the Linux/POSIX pipe write-path change in src/io/PipeWriter.rs (one-page writes per POLLOUT for blocking pipes/FIFOs,… |
Full details: Description check
-
Explanation
The PR description is detailed and directly related to the changeset. It describes the Linux/POSIX pipe write-path change in src/io/PipeWriter.rs (one-page writes per POLLOUT for blocking pipes/FIFOs, send(MSG_DONTWAIT) for sockets, tty unchanged), the IOWriter::bump compaction change, and the new tests in test/js/bun/shell/file-io.test.ts. It also gives reproduction steps, benchmarks, downsides, and alternatives considered. The description is relevant and specific, so the check passes.
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟣
src/io/PipeWriter.rs— Shell callers whose reader closes mid-chunk now getwritten: 0and lose the capture of pages that were already delivered, on every chunk over one page rather than only on a signal-interrupted write. src/io/PipeWriter.rs:132-143 loops write_one_slot_to_pipe once per page; after N pages succeed, the next page's poll reports Hup and sys::write returns EPIPE, and the Err at src/io/PipeWriter.rs:143 discardsoffset. The contract at src/io/PipeWriter.rs:215-218 ('an error here means nothing was written this round') is no longer true for the Pipe arm. Fix: report the pages already written before the error (e.g. return Pending(offset) and surface the error on the next round) so on_write tees them and parents see the true count.Why this was flagged
Trigger: a non-quiet
$writes a chunk larger than bun_core::page_size() to a blocking pipe or FIFO, and the reader exits after consuming part of it. Pages 1..N succeed andoffsetadvances at src/io/PipeWriter.rs:147; page N+1 polls Hup, sys::write gets EPIPE, and line 143 returns WriteResult::Err(err) with offset dropped. on_poll then calls on_error (src/io/PipeWriter.rs:207) without any on_write, so IOWriter::on_error (src/runtime/shell/IOWriter.rs:862) reportswritten: 0for every pending writer and the bytes already in the pipe are never teed into the captured stdout buffer. On the base this needed a partial blocking write(2) (reader close mid-copy); write_to_blocking_pipe wrote the whole chunk in one call, so in the common case either all bytes were counted or none were written. The PR makes the multi-call loop the normal path for every chunk over one page, so the lost count is now the ordinary EPIPE outcome. The doc comment at src/io/PipeWriter.rs:215-218 still asserts the old invariant.Verification: pre-existing. Trigger: a non-quiet
$whose stdout/stderr fd is a blocking pipe/FIFO and whose reader closes after some of a chunk has been written.try_write_with_write_fn(src/io/PipeWriter.rs:132-143, untouched by the diff) advancesoffseton eachOk(wrote)and thenreturn WriteResult::Err(err)withoutoffset. The base already fails the same way by the same route.
A nonblocking descriptor takes the whole write while a reader on the same thread drains the pipe. The blocking descriptor is a todo row: on Linux that write waits in write(2), and the one-slot write of #46307 ends the wait.
|
About the count after EPIPE (
The loss that remains is in |
Problem
$that prints more than a pipe holds sleeps inwrite(2)until the reader drains. This blocks the JS thread. 200 KB into a FIFO read after 2 s: a 100 ms timer fires at 1952 ms.write_to_blocking_pipe(src/io/PipeWriter.rs:252) polls, then writes the whole buffer. POLLOUT proves one free pipe slot.pwritev2(RWF_NOWAIT)(process.stdout/stderr: clear O_NONBLOCK for spawned children, don't drop console output on EAGAIN #43868) does not cover a FIFO or a pre-6.4 pipe.Fix
send(MSG_DONTWAIT). A tty is unchanged.IOWriter::bumpcompacts its queue once half is written. The old rule is quadratic where writes do not wait.test/js/bun/shell/file-io.test.ts, plus 54 related files. Self-reviewed: 64 concerns raised, 53 addressed (Notes).Background
O_NONBLOCKthere breaks children:head -c 1000000dies with EAGAIN.splice(2)relay and a capacity write when FIONREAD is 0. Both keep full speed. process.stdout/stderr: clear O_NONBLOCK for spawned children, don't drop console output on EAGAIN #43868 declined such machinery.Downsides
cat: 1233 ms (main: 649 ms). Another process's output can split a chunk over one page.console.log,Bun.write(Bun.stdout), FreeBSD, and a one-page write that loses a race.Notes
No user report. This was found by reading
write_to_blocking_pipeduring work on #45565 (closed, superseded by #46021). The tracker has no issue for it. #23636 fixed the same symptom in the shell on macOS. In the #43868 review the shell was left alone after a macOS trace of a socket stdout showed each write capped by the kqueue count (discussion). epoll giveson_pollno count (src/io/posix_event_loop.rs:365), so on Linux the cap isusize::MAX.History of the bound. #43868 first sized this write from
F_GETPIPE_SZandFIONREAD(62a5dfd). bb0e24a replaced that with one page after POLLOUT ("Only POLLOUT's guarantee is safe: one free page slot on Linux"), because queued bytes undercount used slots. b2ca90a then dropped the bound ("Drop the per-call nonblocking write machinery") whenprocess.stdoutwent back toO_NONBLOCK, and 2c5f635 kept the per-call writes that cost nothing:pwritev2(RWF_NOWAIT)andsend(MSG_DONTWAIT). The shell's writers got neitherO_NONBLOCKnor the bound. This PR restores the bound for them, with onefstatper writer where bb0e24a had one per write. The socket arm is thesendthat 2c5f635 already uses.Who is affected. The stall needs a shell output chunk larger than the free room of fd 1 or fd 2, a description without
O_NONBLOCK, and:pipe(2)pipe on a kernel withoutFMODE_NOWAITfor pipes (upstream before 6.4), or withBUN_FEATURE_FLAG_DISABLE_RWF_NONBLOCK,child_processor ofBun.spawnwithstdout: "pipe") on a kernel before 6.5,fs/pipe.c,fifo_open) the same holds for a pipe opened by path (/dev/stdout,/proc/self/fd/N, bash> >(cmd)). Not run on a 6.4+ kernel.Checked here on 6.1.155:
pwritev2(RWF_NOWAIT)returns ENOTSUP for a pipe, a FIFO and a socketpair.It covers builtin output, shell messages on stderr, the output of external commands that a non-quiet
$relays, and builtins underbun execand.shfiles. It does not happen with.quiet()or.text(), for a redirect to a FIFO path, or after the script readprocess.stdoutor calledBun.stdout.writer(). Those setO_NONBLOCKon the description until the next spawn with inherited stdio clears it (checked with/proc/self/fdinfo/1).Repro (Linux 6.1.155, release builds of the merge base 71bc818 and of this branch):
/proc/PID/syscallduring the stallwrite(2)epoll_pwait2bun big.sh(mini event loop)write(2)epoll_pwait2Tests. Six new tests in the
output that its reader does not drainblock.a blocking FIFO on stdoutandon stderr(Linux): the fixture is the only reader of the FIFO and reads it on the thread that runs the shell. It checks a write of one slot, of one slot plus one byte and of the capacity plus one byte while exactly one slot is free. Then a small chunk before a chunk larger than the pipe, two shells at once, and 512 KiB from an external command (this reaches the changed branch ofbump, checked with a breakpoint). It checks that the description is still blocking. Last, it makes the description nonblocking and checks that the shell then needs fewerwritecalls than the pipe has slots (syscwin/proc/thread-self/io).two blocking FIFOs under a shell script(Linux):bun two.shrunsseq 1 30000 1>&2 | seq 1 30000on the mini event loop. stdout must arrive while nothing reads stderr.a socketanda socket without RWF_NOWAIT: a child whose stdout is a socketpair prints two small chunks and 4 MiB. It must report 100 event loop turns while that is pending. The second test setsBUN_FEATURE_FLAG_DISABLE_RWF_NONBLOCKso kernels from 6.5 reach thesendarm.a tty: 100 KiB ofechooutput arrives once and in order. It passes with and without the fix and guards the unchanged tty path.a socketpasses there without the fix.file-io,shell-blocking-pipe,exec,epipe,shell-pipe-read-faultandpipeline_stackpass.terminal,spawn,spawn-maxbuf,filesink,process-stdio,bun-write): the only difference is the five new tests. Two failures are on both builds in this container, which has a slow disk:ls > recursive > node_modulestimes out, and 15 to 18 tests ofbun-write.test.jstime out.Costs, measured. Release builds of 71bc818 and this branch, Linux 6.1.155. Syscalls are main-thread counts from a ptrace counter (strace and perf are not installed here), per
await $echo ${p}`` with stdout a FIFO thatcatdrains. A `pipe(2)` pipe gives the same table on this kernel.The same, after the script read
process.stdout(the description hasO_NONBLOCK, main does not stall):fstatis once per writer. Thefcntl(F_GETFL)is once pertry_writecall of more than one page..quiet(): the syscall tables are identical. A tty: one morefstatper writer for a chunk over one page.pipe(2)pipe on Linux 6.4 or later was not measured (no such kernel here). By the code, RWF_NOWAIT returns before any changed line.cat, 11 interleaved runs, median (min to max):O_NONBLOCKO_NONBLOCK$(head -c N /dev/zerointocat), 10 interleaved runs, median (min to max):bumpchange alone: 128 MiB relayed intocatafter the script readprocess.stdout, so no build waits inwrite(2). Main: 5.3 to 6.1 s wall, 3.1 to 3.7 s user CPU. This branch: 1.2 to 1.6 s wall, 0.07 to 0.16 s user CPU (3 runs each).$keeps the whole output forShellOutputin both builds.O_NONBLOCKThe last row shows that main has the same queue wherever its write does not wait.
bumprule keeps a written prefix of up to the size of the unwritten tail before it drops it (main: 128 KB). The two stalled rows above do not show a higher peak from it. A reader at a steady lower speed can hold up to one more tail.write(2)..text+1280 bytes (81,904,741 to 81,906,021), file size unchanged.PosixBufferedWriter(32 bytes) andPosixStreamingWriter(64 bytes) keep their size: the one-byte state replaces the one-byterwf_unsupportedflag.Other visible changes.
console.log, a second$, or the stderr writer under2>&1can land inside a shell chunk. Before, the sleepingwrite(2)kept the chunk whole.process.exit()that races a shell write drops what the pipe has not taken. Before, the process could not exit until the reader drained the pipe.ShellOutput.stdoutholds the bytes of every write round before the one that failed (65536 here). Before it held 0. The exit code is the same. Pages written in the round that fails are not counted:try_write_with_write_fndrops its count on an error, as on main.Not covered. stdout is a FIFO read after 2 s. The number is when a 100 ms timer fires, main then this branch.
console.log(200 KB): 1897 and 1941 ms. The console writer is synchronous by design (b2ca90a).process.stdout.write(200 KB)after aspawnSyncwith inherited stdio: 1838 and 1935 ms. process.stdout/stderr: clear O_NONBLOCK for spawned children, don't drop console output on EAGAIN #43868 keeps Node's behavior there (discussion).await Bun.write(Bun.stdout, 200 KB): 1845 and 1953 ms. It writes on the JS thread. It shares no code with the shell's writer, so it is not part of this PR.sys_pipe.c,pipe_direct_write), a blocking pipe write of 8192 bytes or more waits until the reader took it, whatever the kevent count says. Not run. There is no FreeBSD test lane.on_pollcuts a blocking write to the kqueue byte count.bumpchange applies on every platform.Alternatives.
O_NONBLOCKon the shell's dup, asprocess.stdoutdoes. The description is shared with every child that inherits it. Measured: with the flag set,head -c 1000000 /dev/zeroon that description fails withResource temporarily unavailableafter 65536 bytes, exit 1. A spawn clears the flag for the child, so the shell would have to set it again for each write while that child still runs.is_pollable()in the shell already callsfstaton the dup.IOWriter::__startcould label the poll from it, undercfg(linux, android), andtry_writewould lose itsfstatand itsset_flag. This PR keeps the test in the writer:Flags.is_socketalso decidesno_sigpipe, and Remove libuv on Windows #42819 rewrites that part of the shell.ioctl(FIONREAD)is 0. In an emulation (a hook that changes the length of the write in a release build of this branch) it restores the speed of main: 60 x 1 MiB intocattakes 632 ms on main, 1097 ms with one slot and 603 ms with it, median of 9. Its costs: two more syscall wrappers, a lost race waits for up to the whole pipe and not one slot, and a faulting write by any writer leaves a slot in use while FIONREAD says 0 (seen here on 6.1). It can follow if the exposure is acceptable.robobun/e6242f5e/pipe-splice-relaycopies the bytes into a private pipe and moves them withsplice(SPLICE_F_NONBLOCK), which waits for neither pipe. It removes the 2x cost and the race. It costs 2 fds per thread, 16 bytes per writer and 8.7 KB of.text.Self-review. Two rounds, 64 concerns, 53 addressed. The first round moved the PR from the relay to this bound, because #43868 chose "where a per-call nonblocking write costs nothing" (discussion). The second round added the
O_NONBLOCKcheck, the mini event loop test and the cost rows above. Not done:Bun.write.Related. #43091 makes pipeline commands write into a blocking
pipe(2). It does not change the root stdout and stderr writers. #42819 removes the compaction that thebumphunk changes, so the two conflict.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/shell/file-io.test.ts