Skip to content

spawn: open a Bun.file path once when stdout and stderr name the same path - #46664

Open
robobun wants to merge 2 commits into
mainfrom
robobun/97995c6d/spawn-one-open-stdout-stderr-path
Open

robobun wants to merge 2 commits into
mainfrom
robobun/97995c6d/spawn-one-open-stdout-stderr-path

Conversation

@robobun

@robobun robobun commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Bun.spawn(cmd, { stdout: log, stderr: log }) with one Bun.file(path) loses output in silence. A child writes six lines to stdout and stderr by turns, three remain, and the exit code is 0. Bun.spawnSync does the same.
  • Each path slot gets its own open() (src/spawn_sys/spawn_process.rs:800). The two file offsets both start at 0, so each stream overwrites the other.

Fix

  • When stderr holds the same path bytes as stdout, the stderr slot becomes a duplicate of the stdout slot. POSIX uses dup2(1, 2). Windows uses the same uv_file.
  • Both streams then share one file offset, as with one fd number in both slots.
  • Verified: test/js/bun/spawn/spawn.test.ts, 12 new rows. The 8 same-path rows fail on main. Also bun-types.test.ts and rust:check-all.
  • Self-reviewed: 13 concerns raised, 13 addressed (Notes).

Background

Downsides

  • Still lossy: two spellings of one file, the same path at stdio[3], one path given to two spawns. Notes has each case.
  • Other callers: 0 new syscalls, 0 new allocations. With no file, spawn_process_posix runs 5 more instructions (1,820 to 1,825). Text grows 512 bytes.
Notes

Origin. Found by a fuzz run. No user reported it, so there is no Fixes line. A search of the tracker found no earlier report.

Reproduction (Linux x64).

import fs from "node:fs";
if (process.argv[2] === "child") {
  for (let i = 0; i < 3; i++) {
    fs.writeSync(1, "OUT-" + i + "\n");
    fs.writeSync(2, "ERR-" + i + "\n");
  }
  process.exit(0);
}
const path = process.cwd() + "/log.txt";
const log = Bun.file(path);
await Bun.spawn([process.execPath, import.meta.filename, "child"], { stdout: log, stderr: log }).exited;
console.log(JSON.stringify(fs.readFileSync(path, "utf8")));
// main:    "ERR-0\nERR-1\nERR-2\n"
// this PR: "OUT-0\nERR-0\nOUT-1\nERR-1\nOUT-2\nERR-2\n"
  • 1.4.2, 1.4.3-canary (367d939) and a release build of main (5c6d4e2) all lose the OUT- lines.
  • With lines of unequal length the file is a byte mix of both streams.
  • A child that buffers stdout (python3 print, perl print) loses all of stderr on main: its stdout is flushed at exit, over the stderr lines.

What the change gives. Every byte of both streams is kept. The order in the file is the order of the child's write calls. A python3 or perl child writes stderr first, because it buffers stdout. One fd number in both slots and > file 2>&1 in a shell give the same order. A caller that read the old file saw one stream on top of the other. It now sees both.

This already works with one fd. The change adds no capability. It stops the Bun.file(path) spelling from failing in silence.

const fd = fs.openSync(path, "w");
const proc = Bun.spawn(cmd, { stdout: fd, stderr: fd });
fs.closeSync(fd);

Cases that still lose output (release build of this PR, exit code 0 in each).

Case File after the run Why it is not in this PR
stdout Bun.file(abs), stderr Bun.file("./log.txt"), dir//log.txt, or a symlink err 0\nerr 1\nerr 2\nut line 1\nstdout line 2\n The open runs in the child after chdir. A string normalisation in the parent can merge two different files. An inode compare needs both descriptors in the parent, which is #37641.
One Bun.file at stdout and stdio[3], both written fd3 0\nfd3 1\nfd3 2\nut line 1\nstdout line 2\n A slot from 3 on opens O_RDWR, and Bun cannot tell a reader from a writer. A shared offset breaks a child that reads the file through fd 3 and writes it through fd 1.
One Bun.file at stdout and stderr of two spawns that overlap the six lines of one child, 6 of 12 The two opens belong to two children. Only O_APPEND reaches that case.
  • The same holds for two spawnSync calls in a row: each spawn opens at offset 0. Bun.spawn: truncate stdout/stderr when redirected to Bun.file(path) #35352 covers truncation.
  • stdin is opened read-only, so it never shares the offset of stdout.
  • Two new test rows pin the stdin and stdio[3] cases: a read through fd 0 or fd 3 starts at offset 0.
  • The key is byte equality while the path is opened in the child. The four spellings that are equal bytes work: one object, two objects from one string, a file: URL, .slice(0).

Prior art. systemd does the same for StandardOutput=file:P with StandardError=file:P: can_inherit_stderr_from_stdout() and then dup2(STDOUT_FILENO, fileno) (src/core/exec-invoke.c). Its compare is path_equal. execa documents one file at both options as the way to get one log, and compares path.resolve() of the two (lib/stdio/duplicate.js). Node has no path-typed stdio.

An in-repo sighting. scripts/darwin-ci/lib/tart.ts:25-28 passes one Bun.file as stdout and stderr for the VM boot log. A review comment on #37633 reported that log as overwritten ("verified: only stderr survives"). This PR does not fix that script: the tart hosts run the bun that scripts/darwin-ci/host.sh pins, and a retried boot spawns a second tart run onto the same log. The script needs its own change to one fd.

Measurements (Linux x64, release builds of main 5c6d4e2 and of that commit plus this diff). The merge base is one commit later (6292c65). That commit does not touch src/spawn_sys, src/spawn or bun-spawn.cpp.

  • stdout+stderr on one Bun.file: fd 1 pos 12, fd 2 pos 12, flags 0100001 (base 6 / 6, 0100001). From /proc/self/fdinfo in the child after 6 bytes to each stream. Same for spawnSync and the stdio array form.
  • syscalls per spawnSync, base 5c6d4e2 vs PR: default 22 -> 22, one file 18 -> 18, two files 12 -> 12, same file twice 12 -> 10. Tool: a ptrace tracer (strace is not installed in the build container). It counts the open, dup, close, fcntl, stat, socketpair, fork, exec and wait calls of the JS thread and of the child before exec. It leaves out WNOHANG polls that find the child still running, because their number depends on timing. 3 runs per shape agree. In the same-file shape the child's openat + dup2(8, 2) + close(8) become dup2(1, 2).
  • spawn_process_posix instructions per call, base vs PR: no file 1820 -> 1825, file at stdout 1704 -> 1716, file at stderr 1704 -> 1714, two files 1580 -> 1625, same file twice 1580 -> 1309. Tool: gdb stepi over the third call, callees included (valgrind is not installed). 5 runs per shape. 2 of the 50 runs differ by 86 instructions inside mimalloc. The instructions in the function's own code are the same in all runs: 578 -> 583, 487 -> 499, 487 -> 497, 357 -> 387, 357 -> 385.
  • The 5 instructions with no file are not new source lines. The compiler loads the stdout path slice ahead of the slot loop.
  • allocator calls per spawnSync inside spawn_process_posix, base vs PR: no file 6 -> 6, one file 7 -> 7, two files 7 -> 7, same file twice 7 -> 5.
  • release binary: text +512 bytes (107,635,854 -> 107,636,366); spawn_process_posix 7,498 -> 8,015 bytes (size, nm -S, base 5c6d4e2 vs PR). The file size of bun is the same: 107,808,480 bytes.
  • new rows: 8 of 12 fail on a release build of main and on 1.4.2, 12 of 12 pass on the PR build; all of spawn.test.ts on the PR build: 195 pass, 0 fail.
  • rust:check-all: 0 errors on 12 targets (run before the rebase, on the same two hunks); new rows on CI: Windows 12 of 12 pass on x64 and on aarch64, macOS aarch64 12 of 12 pass. macOS x64 passed 12 of 12 on the first push, with the same src/ hunks. macOS and Windows were not run by hand.

Self-review. 13 concerns, all addressed.

  • Docs (3): the first draft said "in write order" and "Bun compares the paths as written". Both are gone. The docs and the JSDoc now say only what the tests check.
  • Tests (1): added the stdin and stdio[3] rows.
  • Scope (4): the table above lists each case that stays lossy, with its output and the reason. Each is left for a separate change.
  • Origin, prior art, the fd recipe and the tart script (5): the sections above.

Related open PRs.

Test runs (Linux x64, debug build with ASAN, last rebased onto a8fcd31).

  • The 12 new rows pass in every run.
  • Of 7 runs of the whole of test/js/bun/spawn/spawn.test.ts, 4 had no failure. 3 had a 5 s timeout in an existing test: Bun.file() works as stdout in one, stdout can be read in two. stdout can be read starts 10 debug children and takes 1.8 to 4.3 s of its 5 s in the runs that pass. Neither test has a path at stderr, so neither takes the new branch. Each passes alone.
  • The spawnSync rows run one at a time, because spawnSync blocks the test process.
  • test/integration/bun-types/bun-types.test.ts: 24 pass.

no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/spawn/spawn.test.ts

@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between a8fcd31 and bf87d1a.

Files selected for processing: 5
  • docs/runtime/child-process.mdx
  • packages/bun-types/bun.d.ts
  • src/spawn/process.rs
  • src/spawn_sys/spawn_process.rs
  • test/js/bun/spawn/spawn.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Walkthrough

The Windows and POSIX spawn implementations now reuse the stdout file descriptor when stderr targets the same path. Tests cover shared and separate file objects, distinct paths, stderr-only output, and independent read offsets. Runtime and type documentation describe shared output files.

Changes

Shared child output files

Layer / File(s) Summary
Reuse descriptors for matching output paths
src/spawn/process.rs, src/spawn_sys/spawn_process.rs
When stdout and stderr target the same path, both spawn implementations use the already-open stdout descriptor for stderr.
Test and document shared file output
test/js/bun/spawn/spawn.test.ts, docs/runtime/child-process.mdx, packages/bun-types/bun.d.ts
Tests cover shared and separate file objects, distinct paths, stderr-only output, and independent read offsets. Runtime and type documentation describe shared output files.

Suggested reviewers: jarred-sumner

Priority: Normal

Merge Risk: Minimal · up to bf87d

The reviewed change has no identified merge-blocking issue; it is ready for normal merge checks.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: reusing one opened path when stdout and stderr use the same Bun.file path.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description clearly explains the problem, the implementation, limitations, testing, platform coverage, and verification results. It does not use the template headings exactly, but it provides the …
Full details: Description check

Explanation

The description clearly explains the problem, the implementation, limitations, testing, platform coverage, and verification results. It does not use the template headings exactly, but it provides the information required by "What does this PR do?" and "How did you verify your code works?".

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:26 AM PT - Oct 24th, 2026

✅ @robobun, your commit bf87d1a50cf868cd6652cd6c50ef4885d000ab1c passed in Build #131741! 🎉


🧪   To try this PR locally:

bunx bun-pr 46664

That installs a local version of the PR into your bun-46664 executable, so you can run:

bun-46664 --bun

@robobun

robobun commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review.

How it was reproduced (Linux x64): a child writes lines to stdout and stderr by turns with fs.writeSync. It is spawned with one Bun.file(path) as both stdout and stderr. On 1.4.2, 1.4.3-canary and a release build of main, the file holds only the stderr lines (a byte mix when the lines differ in length), and the exit code is 0. With this change the file holds every line of both streams.

The 8 same-path rows in test/js/bun/spawn/spawn.test.ts fail on those builds and pass with this change. The script is in the Notes of the PR body.

CI ran the 12 new rows on Windows x64 and aarch64 and on macOS aarch64. All 12 pass on each, and that includes the stdio[3] rows on Windows. macOS x64 passed all 12 on the first push.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked two things and ruled them out: the POSIX action ordering (slot 1's open is queued before slot 2's dup2(1, 2) in the same loop, and the dup_stdout_to_stderr hack only applies to the Dup2 kind, so stderr cannot dup a not-yet-opened fd 1), and the Windows handle lifetime (the shared uv_file is pushed to uv_files_to_close only once from the stdout slot, so the stderr slot reusing it cannot double-close).

Extended reasoning...

The change touches the POSIX and libuv spawn stdio setup in src/spawn_sys/spawn_process.rs and src/spawn/process.rs, plus .d.ts, docs, and a new describe.each test block in test/js/bun/spawn/spawn.test.ts; it touches no auth, crypto, or input-parsing surface. Two inline findings were already posted (the undocumented Windows skip on the stdio[3] row and the byte-equality-only path compare), so a human look is already signalled; this note only records the ordering and handle-lifetime checks that came up clean.

Comment thread test/js/bun/spawn/spawn.test.ts
Comment thread src/spawn_sys/spawn_process.rs
… path

Each path slot got its own open(), so stdout and stderr had two file
offsets that both started at 0, and each stream overwrote the other.

When stderr holds the same path bytes as stdout, the stderr slot is now a
dup of the stdout slot (dup2(1, 2) on POSIX, the same uv_file on Windows),
so both streams share one open file description.
The Windows spawn code opens a path at an extra slot as well, so the row
does not need a platform skip.
@robobun
robobun force-pushed the robobun/97995c6d/spawn-one-open-stdout-stderr-path branch from ebb7dcd to bf87d1a Compare October 24, 2026 06:50

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed after the second push and found no bugs; the stdio[3] row now runs on Windows, which closes the only nit from my earlier pass. Because this changes how Bun.spawn wires child stdio across both platform backends, a human look at the semantics would still be worthwhile.

What was reviewed:

  • POSIX arm (src/spawn_sys/spawn_process.rs:802): the dup2(1, 2) action is appended after slot 1's open action, and both the Linux action list and the macOS posix_spawn file-actions run in order, so fd 1 already names the file when the dup runs.
  • Windows arm (src/spawn/process.rs:2121): the guard can only match at slot 2 since stdout_path_file is set after slot 1's own arm; the uv_file is pushed to uv_files_to_close once, so no double close. The if-let match guard is not compiled on this Linux host, so I could not confirm it builds beyond the let-chains already used elsewhere in src/.
  • Tests: the describe.each matrix covers both entry points, same object vs same path, distinct paths, stderr-only, and the stdin/stdio[3] negative contract; same-path rows read back exact bytes, so they fail on the base behavior.
Extended reasoning...

The change touches the child-stdio setup in both spawn backends (src/spawn_sys/spawn_process.rs for POSIX, src/spawn/process.rs for Windows libuv), plus docs, the .d.ts JSDoc, and 109 lines of new tests in test/js/bun/spawn/spawn.test.ts; the production diff is about 20 lines. It touches no auth, crypto, or input-parsing surface; the new comparison is a plain byte equality of two user-supplied paths that were already being opened. The second commit (bf87d1a) addressed the only inline nit from the prior run by dropping the Windows skip on the stdio[3] row. Deferring rather than approving because this is a user-visible semantic change to Bun.spawn stdio handling and the Windows arm is cfg-gated code that cannot be type-checked from this host.

@robobun

robobun commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

The Windows arm compiles. The windows x64 - build-bun and windows aarch64 - build-bun lanes built this head in build 131741.

For the human look at the semantics, there is one decision to confirm. When stdout and stderr hold byte-equal paths, the child gets one open file description for both slots, in place of two. The PR body lists what stays as before: stdin, the slots from 3 on, and two different spellings of one file.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants