Skip to content

bun test --isolate, --parallel: emit 'exit' at the end of each node:test file, and fail the file on its exit code - #47431

Draft
robobun wants to merge 2 commits into
mainfrom
robobun/8075abc2/test-isolate-file-exit
Draft

robobun wants to merge 2 commits into
mainfrom
robobun/8075abc2/test-isolate-file-exit

Conversation

@robobun

@robobun robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • bun test --isolate a.test.ts b.test.ts exits 0 with 2 pass, 0 fail when an exit listener of a.test.ts (node:test) stores process.exitCode = 1. So does --parallel.
  • bun test emits exit once, when the run ends. By then swap_global_for_test_isolation (src/jsc/VirtualMachine.rs:6156) has discarded each earlier file's global and its listeners.

Fix

  • TestCommand::run emits exit on a file's process while the file is active. A non-zero exit code it leaves is one failed test, (fail) a.test.ts (exit code 1).
  • process.exit() in such a listener, or in its promise jobs, ends the file, not the process: process.reallyExit unwinds its caller.
  • Verified: test/cli/test/bun-test.test.ts (52 new tests, 51 fail on main), isolation.test.ts, parallel.test.ts, node-test.test.ts.
  • Self-reviewed: 43 concerns raised, 40 addressed as proposed, 3 another way (Notes).

Background

Downsides

Notes

Where this came from

An automated fuzz run found it. No user reported it, and no issue is open for it. #38442 pinned the same gap for --parallel as a limitation, and its review left it as separate work: "Making that work needs listener handling inside isolation". #8434 is a different ask (the exit listeners of a bun:test file), and #38442 decided it the other way. This PR does not change that.

Repro

// a.test.ts
import { test } from "node:test";
let calls = 0;
const mustBeCalledOnce = () => { calls++; };
process.on("exit", () => {
  if (calls !== 1) { console.error(`the check at exit: called ${calls} times, wanted 1`); process.exitCode = 1; }
});
test("a test that forgets to call it", () => { void mustBeCalledOnce; });

// b.test.ts
import { test } from "node:test";
test("a test that passes", () => {});
command main this PR
bun test ./a.test.ts ./b.test.ts exit 1, 0 fail, the check prints after the summary the same
bun test --isolate ./a.test.ts ./b.test.ts exit 0, the check never runs exit 1, 1 fail
bun test --isolate ./b.test.ts ./a.test.ts exit 1, 0 fail exit 1, 1 fail
bun test --parallel=1 ./a.test.ts ./b.test.ts exit 0 exit 1, 1 fail
bun test --parallel=2 ./a.test.ts ./b.test.ts exit 0 exit 1, 1 fail

Output of bun test --isolate ./a.test.ts ./b.test.ts with this PR:

a.test.ts:
(pass) a test that forgets to call it [1.27ms]
the check at exit: called 0 times, wanted 1
(fail) a.test.ts (exit code 1)

b.test.ts:
(pass) a test that passes [1.15ms]

 2 pass
 1 fail
Ran 3 tests across 2 files. [28.00ms]

The rule: the exit status of a process belongs to its owner

Node v26.3.0 has the same two answers. x is the file under test, b passes.

fixture x node --test (a process for each file) node --test --test-isolation=none
listener stores process.exitCode = 1 exit 1, fail 1 exit 1, fail 0
listener calls process.exit(3) exit 1, fail 1 exit 3, fail 0
module scope stores process.exitCode = 5 exit 1, fail 1 exit 5, fail 0
listener throws exit 1, fail 1 exit 1, fail 0
a failed test, and the listener stores 0 exit 1, fail 1 exit 0, fail 1
await 0; process.exit(0) in the listener, and b fails exit 1, fail 1 exit 0, fail 1
a promise job of the listener stores 9 exit 1, fail 1 exit 9, fail 0

Plain bun test gives the right column on main, and this PR does not change it. A file that has its own process now gives the left column.

What happens at the end of a file that owns its global

  • The gate is the one of bun test: only run process.on('exit') listeners when node:test APIs were used #38442: the file called a node:test API, or BUN_TEST_DRAIN_EVENT_LOOP=1 is set. A call is the registration of a test, a suite or a hook, a use of mock, or assert.register. An import is not a call. The mark is now kept for each global. Before, one node:test file opened the gate for every later file of the run. So a bun:test file that follows a node:test file under --isolate no longer has its listeners run.
  • The listeners get exit code 1 when the file has a failed test or an error. Otherwise they get the process.exitCode that this file stored, or 0. A code that an earlier file left in the VM is not this file's.
  • The promise jobs that the listeners queue run before the file ends. A job that stores an exit code, or calls process.exit(), counts for the file. A listener that rejects is an error of the file.
  • The file fails when it leaves a non-zero exit code and has no failure of its own. run() of node:test uses the same rule for its child processes (src/js/node/test.ts).
  • A listener that throws is an Unhandled error between tests of the file: 1 error, exit 1, and no second failure for an exit code.
  • process.exit(N) and process.reallyExit(N) in a listener, or in one of its promise jobs, end the exit with code N. The first call decides the code. The later listeners do not run, and the jobs that are still queued are dropped. A worker stays alive: there is no worker crashed line.
  • A file that has a failed test stays failed when its listener stores 0 or calls process.exit(0). On main such a file, last or alone under --isolate, printed 1 fail and exited 0.
  • After the exit, the VM's exit code goes back to its value at the start of the file.
  • A file that has no exit listener gets no emit. After an emit, process._exiting is false again, because the teardown of the global still runs script that needs process.nextTick. bun test --watch --isolate stays in watch mode, and process.nextTick works in the idle file.
  • The deadline that the file's last test left is removed first. spawnSync in a listener read it and killed its child.
  • A listener that a --preload file sets runs at the end of every such file. With --rerun-each, exit is emitted once, after the last run. --bail counts the failure. The JUnit report gets a failed test case (exit code N) in the file's suite. --dots and --only-failures print the line, and the end of a long run repeats it.

Where the state is

  • Bun::Process has the state of the exit (m_testFileExit), next to m_isExiting. Process__exitTestFile sets it for the emit and for one microtask checkpoint, and then clears it. Process_functionReallyExit reads it. So Bun__Process__exit does not change, and it still never returns on the main thread.
  • The checkpoint is vm.drainMicrotasks() and not GlobalObject::drainMicrotasks(). The second one takes a termination exception as the stop of the VM and forbids script.
  • The exit code is set, read and restored in C++. An earlier version of this branch kept that state in Rust behind a &mut VirtualMachine across the emit. Its release build returned a stale exit code and dropped a store, because script writes the VM through another pointer. Its debug build passed. For the same reason emit_file_exit reads the failure counts again through Jest::runner() after the listeners ran. The test an exit listener that throws or rejects is an error of its file covers that read. I did not build a release binary without it to see that test fail.

Outcome matrix

12 fixtures in 10 modes: 120 runs on release builds of main (9fe1cd6) and of this PR. Three runs of each build give the same table. The debug build of this PR gives the same table as its release build.

  • The fixtures are 6 kinds of file, each as a node:test file and as a bun:test file: a listener that stores process.exitCode = 1, a listener that calls process.exit(3), a file that stores process.exitCode = 5 at module scope, a listener that throws, a listener of a --preload file that stores 7, and a file with a failed test whose listener stores 0.
  • The modes: plain, --isolate (the file first, last, alone, and with --rerun-each=2), --parallel=1, --parallel=2 with one worker for all files (the file first, and last), --parallel=2 with a worker for each file, and --parallel=2 --no-isolate.

Results:

  • 48 cells change. They are the 48 cells of a node:test fixture in a mode that gives the file its own global.
  • 28 go from exit 0 to exit 1. On main 28 of those 48 cells exit 0. With this PR none does.
  • 12 go from exit 3, 5 or 7 to exit 1 with a (fail) line. 8 keep exit 1: the listener now runs, or the file gets its (fail) line.
  • 0 of the 60 bun:test cells change. 0 of the 12 plain cells and 0 of the 12 --no-isolate cells change.
  • No cell has a worker crashed line.
  • With BUN_TEST_DRAIN_EVENT_LOOP=1 the same 48 cells of the bun:test fixtures change too, 28 of them from exit 0 to exit 1. That variable opens the gate for every file, as on main.

The vendored node:test files

111 files under test/js/node/test import node:test. CI runs each in its own process with BUN_TEST_DRAIN_EVENT_LOOP=1, so this PR changes nothing there. I ran each one as bun test --isolate <file> <a passing file> on both release builds:

  • main: 111 exit 0.
  • this PR: 109 exit 0, and 2 exit 1 with (fail) <file> (exit code 1): test-file-write-stream5.js and test-worker-arraybuffer-zerofill.js.
  • Both check with common.mustCall() a callback that runs after the test function returns (a finish event, a message of a worker). node --test passes them, because it waits for the event loop. With BUN_TEST_DRAIN_EVENT_LOOP=1 this PR passes them too.
  • main exits 1 for both when the file runs alone, and when it runs last under --isolate. It exits 0 only when another file follows.

Measurements

Release builds, Linux x64: main 9fe1cd6, and this PR on top of it. The instruction counts are from gdb (nexti for the frame of a function, stepi for a function with its callees).

  • A file whose gate is closed (a bun:test file), for each file:
    • plain bun test: TestCommand::run 441 to 451 instructions in its frame (+10), +2 conditional branches, +0 calls.
    • --isolate: TestCommand::run 438 to 461 (+23), +2 conditional branches, +1 call. The call is emit_file_exit, which returns after 21 instructions (2 conditional branches, and 1 call for the cached value of BUN_TEST_DRAIN_EVENT_LOOP). Together +44.
  • A node:test file that has its own global: emit_file_exit with its callees runs 603 instructions when the file has no exit listener. With one empty listener it runs about 12,400 to 13,000 (two runs).
  • process.reallyExit, which every process.exit() of every program passes: 31 to 34 instructions and 7 to 8 conditional branches before the call of on_exit. +0 calls. Bun__Process__exit does not change.
  • EventEmitter::fireEventListeners with 1, 2 and 3 listeners that do not throw: 165, 253 and 348 instructions to 167, 256 and 352 (+1 for each emit and +1 for each listener). The conditional branches (26, 43, 60) and the calls (2, 3, 5) are equal.
  • Size: .text 108,798,697 to 108,803,817 bytes (+5,120). The file has 138,270,840 and 138,274,936 bytes (+4,096).
  • sizeof(Bun::Process): 120 to 128 bytes. Its cell has 128 bytes in both builds, because JSC sizes cells in steps of 16. TestRunner: +8 bytes (608 in a debug build), one for each process. ExitHandler: 3 bytes in both. These are from debug builds.
  • Frames that a worker sends on fd 3: FileStart has 9 bytes and FileDone 49 in both builds. A file that fails at exit sends one more TestDone frame: 44 bytes for (fail) a.test.ts (exit code 1).
  • 20 node:test files whose exit listener calls process.exit(1), with --parallel=2. main: 0 listeners run, 20 pass, 0 fail, exit 0. This PR: 20 listeners run in 2 worker processes, no worker crashed line, 20 pass, 20 fail, exit 1.
  • BUN_JSC_validateExceptionChecks=1, and BUN_DESTRUCT_VM_ON_EXIT=1 with LeakSanitizer, on the debug build: no report under --isolate, --parallel=1 and --parallel=2. The files: a listener that calls process.exit(), a promise job that calls it, process.reallyExit(), a nested process.emit(), and a listener that rejects.

Not changed, and where it is tracked

Differences from the process of node --test

  • Bun emits exit when the tests and hooks of the file are done. It does not wait for timers, I/O or child processes, and it does not emit beforeExit. That is the moment that bun test already has for the last file. BUN_TEST_DRAIN_EVENT_LOOP=1 waits first. A test pins the difference, and the docs say to do such a check in an after() hook.
  • A file that fails to load emits exit at once, with code 1.
  • No timeout applies to an exit listener, as at the end of a run on main.
  • run({ files }) of node:test gives each file a real process.

Workaround on released versions

A check in an after() hook fails the run in every mode on 1.4.3 (plain, --isolate, --parallel=2, --parallel=2 --no-isolate). It does not help a file that checks in an exit listener and that you cannot change, such as a test that uses node's common.mustCall().

Merge order with #44833

#44833 changes the same function. A test merge with its head b9e43ea has three conflict regions in src/runtime/cli/test_command.rs and one in docs/test/parallel.mdx:

  • the node_test_used initializer (it adds a field next to it),
  • the place of emit_file_exit (it adds functions there),
  • the load-failure arm. bun test: Vitest compatibility #44833 replaces the return with a failed_to_load flag, so the call moves to its loop tail.

One part merges with no conflict and is wrong after the merge: with --isolate=modules the files share one process. file_owns_global is the one place to change. It becomes vm.test_isolation_enabled && !vm.test_isolation_state.shares_global.

Self-review

43 concerns. 40 are addressed as proposed. 3 are addressed in another way:

  • Proposed: keep the file "exited" until its global is gone, and make a later process.exit() from it do nothing. Done instead: the exit covers the emit and the promise jobs, which is what node runs, and after it process.exit() is as on main. A process.exit() that silently returns lets the code after it run. It also hides a process.exit(1) that a close handler of the file calls in the teardown, which ends the run with exit 1 on main.
  • Proposed: one call of the exit in the loop tail, with a failed_to_load flag in place of the return of the load-failure arm. Done instead: one function that holds the gate, called at the two exits. The flag changes what every load failure does after it, and bun test: Vitest compatibility #44833 makes that change.
  • Proposed: a --watch --isolate test. Done instead: a test of process.nextTick in the teardown of a finished file, with and without a listener, and a manual run of watch mode (the listener runs, the session stays, process.nextTick works, a SIGINT handler ends the session).

Suites

  • test/cli/test/bun-test.test.ts: the block process.on('exit') listeners has 62 tests. 51 are new and 1 is rewritten (the test that pinned this gap in bun test: only run process.on('exit') listeners when node:test APIs were used #38442). Debug and release builds of this PR: 62 pass. Release build of main: 11 pass, 51 fail. The new test that passes on main checks that a file with no exit listener keeps process.nextTick in its teardown.
  • The whole bun-test.test.ts passes on the debug build (160 pass) and on the release build.
  • Debug build: isolation.test.ts 50 pass, node-test.test.ts 45 pass, parallel-startup-failure.test.ts 2 pass, isolation-one-module-fetched-twice.test.ts 11 pass, parallel.test.ts 46 pass and 1 fail.
  • The 1 failure is each worker has a unique JEST_WORKER_ID. It depends on how fast three debug workers start, and it also fails in 1 of 3 runs alone. On the release builds of main and of this PR, parallel.test.ts has 47 pass and 0 fail.
  • The 37 test files of this repository that mention node:test, with --parallel=6 on the release builds: 993 pass, 47 skip, 0 fail on both.
  • cargo clippy -p bun_runtime -p bun_jsc --no-deps: no warning.

no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/test/bun-test.test.ts

…est file, and fail the file on its exit code

With --isolate or --parallel each test file has its own global, and so its
own process object. bun test emitted 'exit' once, at the end of the run, on
the global of the last file. The exit listeners of every other file were
discarded with their global, so a failed check at exit (node's
common.mustCall()) passed.

TestCommand::run now emits 'exit' on the process of a file that owns its
global while the file is still active. The listeners get 1 when the file
has failed, else the exit code that the file stored. The promise jobs of
the listeners run as part of the exit. A non-zero exit code that the file
leaves is one failed test, "(fail) <file> (exit code N)".

process.exit() in such a listener, or in one of its promise jobs, ends the
exit of the file and not the process. The state for that is on the file's
own Process object, and process.reallyExit reads it.

The mark "a node:test API was called" is now kept for each global, so a
bun:test file that follows a node:test file under --isolate no longer has
its exit listeners run. Plain bun test and --parallel --no-isolate do not
change.
@robobun

robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:08 AM PT - Oct 30th, 2026

❌ @robobun, your commit 3475a98 has 4 failures in Build #135628 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47431

That installs a local version of the PR into your bun-47431 executable, so you can run:

bun-47431 --bun

@robobun

robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status of #47431

Reproduced on main (9fe1cd6, release build, Linux x64) with the two files from the Notes of the PR body:

  • bun test --isolate ./a.test.ts ./b.test.ts exits 0 with 2 pass, 0 fail. The check at exit of a.test.ts never runs.
  • bun test --parallel=1 ./a.test.ts ./b.test.ts and bun test --parallel=2 ./a.test.ts ./b.test.ts exit 0 too.
  • bun test ./a.test.ts ./b.test.ts exits 1, and so does node --test.

With this PR the three commands exit 1 with (fail) a.test.ts (exit code 1).

This PR is a draft. It changes TestCommand::run, which #44833 also changes, and the order of the two is open.

CI on 3475a98 (build 135628, finished): test/cli/test/bun-test.test.ts fails on no lane. Its 166 tests ran on the ASAN, Windows x64, Windows aarch64, macOS aarch64 and macOS x64 jobs. The red tests are not touched by this PR:

  • test/aot/engine/ir-6.test.ts is also red on main (build 135526).
  • Three comparisons under test/aot/parallel/node-http/ failed on macOS. The same three fail in builds of other branches (135642, 135662, 135708, 135718), and they passed on the source of this PR in build 135610.

… on Windows

On Windows the pipe to the coordinator stays referenced, so the event loop
of a worker is always alive and the drain of BUN_TEST_DRAIN_EVENT_LOOP=1
(VirtualMachine::on_before_exit) never ends there. That is so on main too.
The --isolate case of the same test runs on Windows.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants