Skip to content

bun test: run every callback when native code leaves an exception pending - #44491

Draft
robobun wants to merge 4 commits into
mainfrom
robobun/54dc25df/test-runner-pending-exception
Draft

robobun wants to merge 4 commits into
mainfrom
robobun/54dc25df/test-runner-pending-exception

Conversation

@robobun

@robobun robobun commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • bun test prints (pass) for hooks, tests and describe bodies it never called, once native code returns with a JS exception still pending. With retry, a failing run exits 0. Debug builds abort: panic: a task returned Ok with a JS exception pending.
  • Cause: EventLoop::run_callback_with_result_and_forcefully_drain_microtasks (src/jsc/event_loop.rs:1262) does not call over a pending exception and returns Ok(undefined). BunTest::run_test_callback, its only caller, reads that as the callback's return.
  • Reach: a zod 3.24.2 error for a cyclic input. Its message getter throws while bun prints it.

Fix

  • The runner takes a pending exception after each error it prints, before each callback and at each file end. The callback then runs.
  • The runner calls the callback directly. The helper had no other caller and is deleted.
  • Verified: test/js/bun/test/failure-skip.test.ts (11 new tests fail on main), expect-extend.test.js. Also test/js/bun/test and test/cli/test.
  • Self-review: the review run stopped before its verdict. Its open points were checked by hand (Notes).

Background

  • A thrown exception stays pending on the VM until native code takes it.
  • The error printer runs user code (getters, inspect hooks) and returns nothing.
  • Considered Err(Thrown) from the helper: a never-called test.failing prints (pass). Considered a check after prints only: a Bun.serve exception still skips callbacks.

Downsides

  • An exception left outside the runner is reported as # Unhandled error between tests. A run that exited 0 with skipped tests can exit 1.
  • A printed failure costs 15 more instructions (108 to 123), a passing callback 44 fewer (221 to 177). Release .text is unchanged: 58,187,957 bytes.
Notes

Repros. Release builds. 1.4.3-canary.1 and main give the same results.

// hooks.test.js
import { test, expect, beforeEach, afterEach, afterAll } from "bun:test";
class E extends Error { get message() { throw new TypeError("a bug in the getter"); } }
beforeEach(() => console.log("beforeEach ran"));
afterEach(() => console.log("afterEach ran"));
afterAll(() => console.log("afterAll ran"));
test("first", () => { console.log("first body ran"); throw new E(); });
test("second", () => { console.log("second body ran"); expect(1).toBe(2); });
test("third", async () => { console.log("third body ran"); await Promise.reject(new Error("x")); });
  • main: (fail) first, (pass) second, (pass) third, 2 pass 1 fail. Only beforeEach ran and first body ran print.
  • This branch: every hook and body runs, 0 pass 3 fail.
// green.test.js (class E as above)
let n = 0;
test("flaky", () => { if (n++ === 0) throw new E(); }, { retry: 2 });
test("must fail", () => { console.log("BODY RAN"); expect(1).toBe(2); });
  • main: (pass) flaky (attempt 2), (pass) must fail, 2 pass 0 fail, exit 0. BODY RAN never prints.
  • This branch: 1 pass 1 fail, exit 1.

Other results on main:

  • A describe body that follows one that throws such an error is not called. Its tests are not collected.
  • Two files, the exception left at the last callback of the first file: exit 1, no summary line, the second file never loads. With --isolate the process aborts.
  • A concurrent test that is in flight never resumes and times out. Microtasks do not drain while an exception is pending.
  • zod 3.24.2: z.literal("expected").parse(cyclic) with cyclic.self = cyclic. The message getter of ZodError calls JSON.stringify on the issues. The next test prints (pass) and its body never runs.
  • Bun.serve with a fetch handler that returns a value that is not a Response, and whose inspect hook throws. The response path of the server (render_missing_invalid_response) leaves the exception, not a print of the runner. A file with one failing test gives 2 pass 0 fail, exit 0. The next (done) => test gets done === undefined.

The three places.

  • After a print: print_error in bun_test.rs wraps the three run_error_handler calls of the runner (BunTest::on_uncaught_exception, on_unhandled_rejection with no active file, a rejected async matcher in expect.rs). The exception prints under the same report. If that second print throws again, that exception is taken and one note: line replaces it. A termination is left for BunTest::run.
  • Before a callback: the first statement of run_test_callback, above the bind of done. Without that order the bind meets the exception, charges it to the entry, and calls the body with done undefined. The exception is reported as # Unhandled error between tests.
  • End of a file: in TestCommand::run before handle_rejected_promises, through report_error_or_terminate. That also runs the microtask checkpoint that the pending exception held back.

Measurements. Release builds of f4d755a with and without this diff. Instruction counts are gdb stepi counts inside the named functions. The numbers predate a later 2-line change in the cold function print_exception_left_by_print.

  • Passing runner callback: 177 instructions, 25 conditional branches (main 221 / 27). Counted over run_test_callback, the helper, drain_microtasks_with_global and drain_quic_if_necessary. 10 identical callbacks.
  • run_callback / run_callback_with_result: 661 / 690 bytes on both builds. The disassembly is equal except one relocated data address.
  • Size: .text is 58,187,957 bytes and the stripped binary is 80,873,032 bytes on both builds. Function symbols: helper -911 B, run_test_callback +263 B, on_uncaught_exception 643 B to 311 B plus 576 B (print_uncaught_exception), three cold functions +334 B, TestCommand::run +80 B, execute_custom_matcher +127 B. 0 host functions.
  • Printed failure with nothing left pending: 123 instructions (main 108), 20 conditional branches (main 22). For 50 failing tests: run_error_handler 50 vs 50 calls, write 98 vs 98 syscalls.
  • 150 passing callbacks: mimalloc allocations 1856 to 1866 (main 1857 to 1859, four runs each), queued runner tasks 0 vs 0, write syscalls 49 vs 49. The three new cold paths run 0 times.
  • Per exception left by a print: +1 run_error_handler call, +1 write. hooks.test.js: 10 of 10 callbacks called (main 2 of 10). Per file: +3 instructions.
  • Existing suites on a debug build: the new cold paths run 0 times across the 106 files of test/js/bun/test, test/cli/test and test/js/node/test_runner. 104 files pass. pretty-format-overflow.test.ts and snapshot-tests/snapshots/snapshot.test.ts fail the same way on main.
  • Against a control where the getter or inspect hook does not throw (14 fixtures, also --isolate and --parallel): 180 of 180 release runs and 54 of 54 debug runs give the callback order, verdict lines and pass/fail counts of the control. 70 and 21 of them (the Bun.serve source) add 1 error. One fixture where every test passes goes from exit 0 to exit 1. Debug aborts: 0 of 16 (main 9 of 16). JUnit: 3 testcase / 3 failure (control 3 / 3). GitHub annotations: one more ::error line per exception.
  • Not available in the container: valgrind, perf, strace, bloaty. gdb gave the instruction and syscall counts.

Tests.

Self-review. The review run stopped before its verdict. Checks done by hand on its open points:

  • Each place is needed. Debug builds with one place disabled: without the take after a print, the 8 print: tests and the expect-extend test fail. Without the take before a callback, server: the next callback runs fails. Without the take at file end, both server: the next file runs tests fail.
  • The new tests pass with BUN_JSC_validateExceptionChecks=1.
  • The new checks use has_exception(), as enter_js and release_task_unrun do.
  • With a fetch() client in place of the raw socket, a debug build of this branch gives the control plus 1 error (3 of 3 runs). On 1.4.3-canary.1 the same file is 3 pass 0 fail, exit 0, in 2 of 3 runs.

Left as is.

@github-actions github-actions Bot added the claude label Oct 3, 2026
@robobun

robobun commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:08 PM PT - Oct 18th, 2026

✅ @robobun, your commit 34cd03a2f157a398c9ad98a29d991b9926c69a39 passed in Build #128443! 🎉


🧪   To try this PR locally:

bunx bun-pr 44491

That installs a local version of the PR into your bun-44491 executable, so you can run:

bun-44491 --bun

@robobun

robobun commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Status

Reproduced on 1.4.3-canary.1 and on main, release builds:

// bun test ./green.test.js
import { test, expect } from "bun:test";
class E extends Error { get message() { throw new TypeError("a bug in the getter"); } }
let n = 0;
test("flaky", () => { if (n++ === 0) throw new E(); }, { retry: 2 });
test("must fail", () => { console.log("BODY RAN"); expect(1).toBe(2); });
  • main: (pass) flaky (attempt 2), (pass) must fail, 2 pass 0 fail, exit 0. BODY RAN never prints.
  • This PR: BODY RAN, (fail) must fail, 1 pass 1 fail, exit 1.

The PR is a draft: its self-review did not complete. The body lists the checks that were done by hand.

…next callback

The runner called each hook, test body and describe body through
EventLoop::run_callback_with_result_and_forcefully_drain_microtasks. With
an exception pending that helper did not call the callback and returned
Ok(undefined), which the runner read as the callback's own return: the
entry was reported as a pass without running.

The runner now takes such an exception in three places: after each of its
prints of an error (the printer runs the error's own accessors), before
each callback, and at the end of each file. The helper had one caller. It
is deleted and the call is made directly.
@robobun
robobun force-pushed the robobun/54dc25df/test-runner-pending-exception branch from 6b4ebfe to bfbec5b Compare October 17, 2026 07:46
Comment thread src/runtime/cli/test_command.rs Outdated
Comment thread src/runtime/test_runner/bun_test.rs Outdated
Comment thread src/runtime/test_runner/bun_test.rs Outdated
Comment thread src/runtime/test_runner/bun_test.rs Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant