Repository navigation
Conversation
…est file, and fail the file on its exit code With --isolate or --parallel each test file has its own global, and so its own process object. bun test emitted 'exit' once, at the end of the run, on the global of the last file. The exit listeners of every other file were discarded with their global, so a failed check at exit (node's common.mustCall()) passed. TestCommand::run now emits 'exit' on the process of a file that owns its global while the file is still active. The listeners get 1 when the file has failed, else the exit code that the file stored. The promise jobs of the listeners run as part of the exit. A non-zero exit code that the file leaves is one failed test, "(fail) <file> (exit code N)". process.exit() in such a listener, or in one of its promise jobs, ends the exit of the file and not the process. The state for that is on the file's own Process object, and process.reallyExit reads it. The mark "a node:test API was called" is now kept for each global, so a bun:test file that follows a node:test file under --isolate no longer has its exit listeners run. Plain bun test and --parallel --no-isolate do not change.
Collaborator
Author
|
Updated 2:08 AM PT - Oct 30th, 2026
❌ @robobun, your commit 3475a98 has 4 failures in
🧪 To try this PR locally: bunx bun-pr 47431That installs a local version of the PR into your bun-47431 --bun |
Collaborator
Author
|
Status of #47431 Reproduced on main (9fe1cd6, release build, Linux x64) with the two files from the Notes of the PR body:
With this PR the three commands exit 1 with This PR is a draft. It changes CI on 3475a98 (build 135628, finished):
|
… on Windows On Windows the pipe to the coordinator stays referenced, so the event loop of a worker is always alive and the drain of BUN_TEST_DRAIN_EVENT_LOOP=1 (VirtualMachine::on_before_exit) never ends there. That is so on main too. The --isolate case of the same test runs on Windows.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
bun test --isolate a.test.ts b.test.tsexits 0 with2 pass, 0 failwhen an exit listener ofa.test.ts(node:test) storesprocess.exitCode = 1. So does--parallel.bun testemitsexitonce, when the run ends. By thenswap_global_for_test_isolation(src/jsc/VirtualMachine.rs:6156) has discarded each earlier file's global and its listeners.Fix
TestCommand::runemitsexiton a file'sprocesswhile the file is active. A non-zero exit code it leaves is one failed test,(fail) a.test.ts (exit code 1).process.exit()in such a listener, or in its promise jobs, ends the file, not the process:process.reallyExitunwinds its caller.test/cli/test/bun-test.test.ts(52 new tests, 51 fail on main),isolation.test.ts,parallel.test.ts,node-test.test.ts.Background
--isolategives each file a fresh global, so its ownprocess.--parallelimplies it.node:testAPI. Its review left this gap: "needs listener handling inside isolation".Downsides
--isolateor--parallelrun when it leaves a non-zero exit code. Bun does not wait for timers: 2 of 111 vendorednode:testfiles fail thatnode --testpasses.bun test+10 instructions,--isolate+44. Perprocess.exit(): +3. Release.text+5,120 bytes.--parallel --no-isolate(bun test --parallel: fail the run when a worker is killed or exits non-zero outside a test file #44431).Notes
Where this came from
An automated fuzz run found it. No user reported it, and no issue is open for it. #38442 pinned the same gap for
--parallelas a limitation, and its review left it as separate work: "Making that work needs listener handling inside isolation". #8434 is a different ask (the exit listeners of abun:testfile), and #38442 decided it the other way. This PR does not change that.Repro
bun test ./a.test.ts ./b.test.ts0 fail, the check prints after the summarybun test --isolate ./a.test.ts ./b.test.ts1 failbun test --isolate ./b.test.ts ./a.test.ts0 fail1 failbun test --parallel=1 ./a.test.ts ./b.test.ts1 failbun test --parallel=2 ./a.test.ts ./b.test.ts1 failOutput of
bun test --isolate ./a.test.ts ./b.test.tswith this PR:The rule: the exit status of a
processbelongs to its ownerNode v26.3.0 has the same two answers.
xis the file under test,bpasses.xnode --test(a process for each file)node --test --test-isolation=noneprocess.exitCode = 1fail 1fail 0process.exit(3)fail 1fail 0process.exitCode = 5fail 1fail 0fail 1fail 0fail 1fail 1await 0; process.exit(0)in the listener, andbfailsfail 1fail 1fail 1fail 0Plain
bun testgives the right column on main, and this PR does not change it. A file that has its ownprocessnow gives the left column.What happens at the end of a file that owns its global
node:testAPI, orBUN_TEST_DRAIN_EVENT_LOOP=1is set. A call is the registration of a test, a suite or a hook, a use ofmock, orassert.register. An import is not a call. The mark is now kept for each global. Before, onenode:testfile opened the gate for every later file of the run. So abun:testfile that follows anode:testfile under--isolateno longer has its listeners run.process.exitCodethat this file stored, or 0. A code that an earlier file left in the VM is not this file's.process.exit(), counts for the file. A listener that rejects is an error of the file.run()ofnode:testuses the same rule for its child processes (src/js/node/test.ts).Unhandled error between testsof the file:1 error, exit 1, and no second failure for an exit code.process.exit(N)andprocess.reallyExit(N)in a listener, or in one of its promise jobs, end the exit with code N. The first call decides the code. The later listeners do not run, and the jobs that are still queued are dropped. A worker stays alive: there is noworker crashedline.process.exit(0). On main such a file, last or alone under--isolate, printed1 failand exited 0.process._exitingisfalseagain, because the teardown of the global still runs script that needsprocess.nextTick.bun test --watch --isolatestays in watch mode, andprocess.nextTickworks in the idle file.spawnSyncin a listener read it and killed its child.--preloadfile sets runs at the end of every such file. With--rerun-each,exitis emitted once, after the last run.--bailcounts the failure. The JUnit report gets a failed test case(exit code N)in the file's suite.--dotsand--only-failuresprint the line, and the end of a long run repeats it.Where the state is
Bun::Processhas the state of the exit (m_testFileExit), next tom_isExiting.Process__exitTestFilesets it for the emit and for one microtask checkpoint, and then clears it.Process_functionReallyExitreads it. SoBun__Process__exitdoes not change, and it still never returns on the main thread.vm.drainMicrotasks()and notGlobalObject::drainMicrotasks(). The second one takes a termination exception as the stop of the VM and forbids script.&mut VirtualMachineacross the emit. Its release build returned a stale exit code and dropped a store, because script writes the VM through another pointer. Its debug build passed. For the same reasonemit_file_exitreads the failure counts again throughJest::runner()after the listeners ran. The testan exit listener that throws or rejects is an error of its filecovers that read. I did not build a release binary without it to see that test fail.Outcome matrix
12 fixtures in 10 modes: 120 runs on release builds of main (9fe1cd6) and of this PR. Three runs of each build give the same table. The debug build of this PR gives the same table as its release build.
node:testfile and as abun:testfile: a listener that storesprocess.exitCode = 1, a listener that callsprocess.exit(3), a file that storesprocess.exitCode = 5at module scope, a listener that throws, a listener of a--preloadfile that stores 7, and a file with a failed test whose listener stores 0.--isolate(the file first, last, alone, and with--rerun-each=2),--parallel=1,--parallel=2with one worker for all files (the file first, and last),--parallel=2with a worker for each file, and--parallel=2 --no-isolate.Results:
node:testfixture in a mode that gives the file its own global.(fail)line. 8 keep exit 1: the listener now runs, or the file gets its(fail)line.bun:testcells change. 0 of the 12 plain cells and 0 of the 12--no-isolatecells change.worker crashedline.BUN_TEST_DRAIN_EVENT_LOOP=1the same 48 cells of thebun:testfixtures change too, 28 of them from exit 0 to exit 1. That variable opens the gate for every file, as on main.The vendored
node:testfiles111 files under
test/js/node/testimportnode:test. CI runs each in its own process withBUN_TEST_DRAIN_EVENT_LOOP=1, so this PR changes nothing there. I ran each one asbun test --isolate <file> <a passing file>on both release builds:(fail) <file> (exit code 1):test-file-write-stream5.jsandtest-worker-arraybuffer-zerofill.js.common.mustCall()a callback that runs after the test function returns (afinishevent, a message of a worker).node --testpasses them, because it waits for the event loop. WithBUN_TEST_DRAIN_EVENT_LOOP=1this PR passes them too.--isolate. It exits 0 only when another file follows.Measurements
Release builds, Linux x64: main 9fe1cd6, and this PR on top of it. The instruction counts are from gdb (
nextifor the frame of a function,stepifor a function with its callees).bun:testfile), for each file:bun test:TestCommand::run441 to 451 instructions in its frame (+10), +2 conditional branches, +0 calls.--isolate:TestCommand::run438 to 461 (+23), +2 conditional branches, +1 call. The call isemit_file_exit, which returns after 21 instructions (2 conditional branches, and 1 call for the cached value ofBUN_TEST_DRAIN_EVENT_LOOP). Together +44.node:testfile that has its own global:emit_file_exitwith its callees runs 603 instructions when the file has no exit listener. With one empty listener it runs about 12,400 to 13,000 (two runs).process.reallyExit, which everyprocess.exit()of every program passes: 31 to 34 instructions and 7 to 8 conditional branches before the call ofon_exit. +0 calls.Bun__Process__exitdoes not change.EventEmitter::fireEventListenerswith 1, 2 and 3 listeners that do not throw: 165, 253 and 348 instructions to 167, 256 and 352 (+1 for each emit and +1 for each listener). The conditional branches (26, 43, 60) and the calls (2, 3, 5) are equal..text108,798,697 to 108,803,817 bytes (+5,120). The file has 138,270,840 and 138,274,936 bytes (+4,096).sizeof(Bun::Process): 120 to 128 bytes. Its cell has 128 bytes in both builds, because JSC sizes cells in steps of 16.TestRunner: +8 bytes (608 in a debug build), one for each process.ExitHandler: 3 bytes in both. These are from debug builds.FileStarthas 9 bytes andFileDone49 in both builds. A file that fails at exit sends one moreTestDoneframe: 44 bytes for(fail) a.test.ts (exit code 1).node:testfiles whose exit listener callsprocess.exit(1), with--parallel=2. main: 0 listeners run,20 pass, 0 fail, exit 0. This PR: 20 listeners run in 2 worker processes, noworker crashedline,20 pass, 20 fail, exit 1.BUN_JSC_validateExceptionChecks=1, andBUN_DESTRUCT_VM_ON_EXIT=1with LeakSanitizer, on the debug build: no report under--isolate,--parallel=1and--parallel=2. The files: a listener that callsprocess.exit(), a promise job that calls it,process.reallyExit(), a nestedprocess.emit(), and a listener that rejects.Not changed, and where it is tracked
--parallel --no-isolate: a worker keeps one global, so there is no end of a file. The listeners run when the worker exits, and their exit code is dropped (bun test --parallel: fail the run when a worker is killed or exits non-zero outside a test file #44431). An error that fires there is bun test --parallel: count errors that fire while a worker has no test file #44841.bun test: a listener that throws leaves exit code 0 (bun test: exit 1 when a process.on('exit') listener throws #41588). A listener that stores 0 after a failed test leaves exit code 0 (question 3 of bun test: report a run that has already failed, and do not exit 0, when process.exit() ends it #44503). Node without isolation does the same.--bailin every mode: an exit listener that stores 0 turnsBailed out after 1 failureinto exit 0 (bun test --bail: keep exit code 1 when an 'exit' listener stores 0 #46639).process.exit()itself: a serial run ends at once, and a worker reports the file as crashed (bun test: report a run that has already failed, and do not exit 0, when process.exit() ends it #44503). Aprocess.exit()that the teardown of a finished file calls is this case too. It ends the run as on main.bun:testfile that storesprocess.exitCode: the run exits with that code when the file runs in the main process, and with 0 when workers run the files. No PR has it yet.process.emit()takes the unwind of aprocess.exit()that one of its listeners calls. So an exit listener that callsprocess.emit()goes on after it. The exit code of the file stays the one of the firstprocess.exit(), and a secondprocess.exit()unwinds again: a test pins both. worker: take a listener's termination at the native boundary, end the tick pass on a stopped VM #45298, worker: keep a TerminationException pending through microtask jobs, event listeners, and emit() #40128 and fix(process): propagate synchronous emit listener exceptions #45021 are about that swallow.(fail)line of its file, because the line and the output travel on different pipes (bun test --parallel: print a test's error output before its result line #41639).process.exitCode = undefineddoes not clear the code in Bun (fix(process): allow clearing exitCode #42607).BUN_TEST_DRAIN_EVENT_LOOP=1with--parallelon Windows: a worker does not finish its first file. The pipe to the coordinator keeps the worker's event loop alive, andon_before_exitwaits for that loop (src/runtime/cli/test/parallel/Channel.rs). That call is before the new code. The test of this variable is skipped for the worker case on Windows, and runs there for--isolate. No PR has it yet.Differences from the process of
node --testexitwhen the tests and hooks of the file are done. It does not wait for timers, I/O or child processes, and it does not emitbeforeExit. That is the moment thatbun testalready has for the last file.BUN_TEST_DRAIN_EVENT_LOOP=1waits first. A test pins the difference, and the docs say to do such a check in anafter()hook.exitat once, with code 1.run({ files })ofnode:testgives each file a real process.Workaround on released versions
A check in an
after()hook fails the run in every mode on 1.4.3 (plain,--isolate,--parallel=2,--parallel=2 --no-isolate). It does not help a file that checks in an exit listener and that you cannot change, such as a test that uses node'scommon.mustCall().Merge order with #44833
#44833 changes the same function. A test merge with its head b9e43ea has three conflict regions in
src/runtime/cli/test_command.rsand one indocs/test/parallel.mdx:node_test_usedinitializer (it adds a field next to it),emit_file_exit(it adds functions there),returnwith afailed_to_loadflag, so the call moves to its loop tail.One part merges with no conflict and is wrong after the merge: with
--isolate=modulesthe files share oneprocess.file_owns_globalis the one place to change. It becomesvm.test_isolation_enabled && !vm.test_isolation_state.shares_global.Self-review
43 concerns. 40 are addressed as proposed. 3 are addressed in another way:
process.exit()from it do nothing. Done instead: the exit covers the emit and the promise jobs, which is what node runs, and after itprocess.exit()is as on main. Aprocess.exit()that silently returns lets the code after it run. It also hides aprocess.exit(1)that a close handler of the file calls in the teardown, which ends the run with exit 1 on main.failed_to_loadflag in place of thereturnof the load-failure arm. Done instead: one function that holds the gate, called at the two exits. The flag changes what every load failure does after it, and bun test: Vitest compatibility #44833 makes that change.--watch --isolatetest. Done instead: a test ofprocess.nextTickin the teardown of a finished file, with and without a listener, and a manual run of watch mode (the listener runs, the session stays,process.nextTickworks, a SIGINT handler ends the session).Suites
test/cli/test/bun-test.test.ts: the blockprocess.on('exit') listenershas 62 tests. 51 are new and 1 is rewritten (the test that pinned this gap in bun test: only run process.on('exit') listeners when node:test APIs were used #38442). Debug and release builds of this PR: 62 pass. Release build of main: 11 pass, 51 fail. The new test that passes on main checks that a file with no exit listener keepsprocess.nextTickin its teardown.bun-test.test.tspasses on the debug build (160 pass) and on the release build.isolation.test.ts50 pass,node-test.test.ts45 pass,parallel-startup-failure.test.ts2 pass,isolation-one-module-fetched-twice.test.ts11 pass,parallel.test.ts46 pass and 1 fail.each worker has a unique JEST_WORKER_ID. It depends on how fast three debug workers start, and it also fails in 1 of 3 runs alone. On the release builds of main and of this PR,parallel.test.tshas 47 pass and 0 fail.node:test, with--parallel=6on the release builds: 993 pass, 47 skip, 0 fail on both.cargo clippy -p bun_runtime -p bun_jsc --no-deps: no warning.no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/test/bun-test.test.ts