Repository navigation
chore(sync): preserve upstream history and repair WebKit consumers - #221
Draft
vincentkoc wants to merge 101 commits into
Draft
vincentkoc wants to merge 101 commits into
vincentkoc wants to merge 101 commits into
Conversation
### What does this PR do? Moves `WEBKIT_VERSION` from `5e3554face13` to `680ef47cec2c`, the head of oven-sh/WebKit `main`. The `autobuild-680ef47c…` release is published. What the fork's `main` gained in that range: - oven-sh/WebKit#889: merge of upstream [`f59345e04a8d`](WebKit/WebKit@f59345e) (2026-10-22). 193 upstream commits since `33ce3bf11a71`, 37 of them in JavaScriptCore, WTF or bmalloc. - oven-sh/WebKit#892: a completely full block that the allocator stumbles on is recorded as allocated. - oven-sh/WebKit#880: the control flow profiler counts the parts of a loop header. - oven-sh/WebKit#807: `BytecodeOptimizer` aborted when jump offsets settled one pass before the forced widening. - oven-sh/WebKit#881: CI only (sha256 for toolchain downloads, FreeBSD 14.4, which is what Bun already targets). The Android NDK is unchanged. `JSType.h`, the WebCore bindings code generator and the release tarball names did not change. #### Changes in Bun besides the version **BIGINT32 is gone upstream** ([`f59345e04a`](WebKit/WebKit@f59345e)). It was never enabled (`USE_BIGINT32` was `0`), so none of this changes behaviour, but Bun does not compile without it: - `JSBigInt::makeHeapBigIntOrBigInt32(g, v)` → `JSBigInt::createFrom(g, v)`, which is what it expanded to (`NodeSqlite.cpp`, `JSBuffer.cpp`, `JSCTestingHelpers.cpp`). Every call site passes an `int64_t` or a `uint64_t`, so the overload is exact. - `JSValue::isBigInt32()` always returned `false`. Removed: `JSC__JSValue__isBigInt32` and `JSValue::is_big_int32`, their uses in `toBeEven` / `toBeOdd`, an `ASSERT` in `JSC__JSValue__asBigIntCompare`, and the test bindgen emitted for a `bigint` that is not a cell. - `tryConvertToBigInt32(bigInt)` → `bigInt` in `SerializedScriptValue.cpp`. - The `#if USE(BIGINT32)` blocks in `bindings.cpp`, `SerializedScriptValue.cpp` and `napi.cpp`, which call functions that no longer exist. - The `JSGlobalObject*` of `JSC__JSValue__asBigIntCompare`, which only one of those blocks read, and with it the `global` of `JSValue::as_big_int_compare` and of the test runner's wrapper. **Loop headers have basic blocks of their own** (oven-sh/WebKit#880): - `JSInspectorProfiler.cpp` leaves out the blocks with `m_isInLoopHeader`. V8 has no range for them, and what reads `Profiler.takePreciseCoverage` is written for V8's ranges. - `source-positions-corpus.ts`: the count of nine loop header lines. `for (let i = 0; i < 3; i++) {` is 4, how often the condition ran, and was 1. Both are the same hunks as in oven-sh#46534, so they merge cleanly with it. `WEBKIT_VERSION` will conflict there: oven-sh#46534 pins `e450c494a84f`, which this contains. **`test/bundler/bundler_bytecode_portable.test.ts`**: the snapshot. Removing `op_is_big_int` renumbers the opcodes after it, so the serialized bytecode changed on every platform. 24 hashes and 13 sizes of `.jsc` payloads change, and no hash of the printed JavaScript. **`test/js/bun/jsc/webkit-upgrade-f59345e04a.test.ts`** (new): pins five of the engine changes below, like the files for the earlier syncs. #### Other upstream changes that reach Bun Checked against Bun's sources, no change needed: - `numberToShortestStringBufferSize` is 39, was 34. Bun passes the 124-byte `NumberToStringBuffer`. - `JSObject::forEachOwnIndexedProperty` lost its `JSGlobalObject*`, `immutableButterflyToStringCache` moved from `Heap` to `VM`, the inline parts of `HasOwnPropertyCache` moved to `HasOwnPropertyCacheInlines.h`, the `swapCodeRefForDebugger` family moved to `NativeExecutable`. Bun uses none of them. - `JSBoundFunction::create` gained an overload that takes a `SourceTaintedOrigin`. The one that takes a `SourceCode`, which Bun calls, is still there. - `SpecDataViewObject` is bit 49, was 50. Bun has no copy of `SpeculatedType`. - The bytecode cache key is a hash of the WebKit commit, so a cache written before the opcodes were renumbered is not read. Behaviour that changes with the engine: - In the head of a `for`: a destructuring declaration without an initializer is a SyntaxError, `let` is an identifier in sloppy code, and `for await (async of` is allowed ([`926bafc4e`](WebKit/WebKit@926bafc4e)). - An overwritten `length` of a sloppy arguments object is converted with `ToLength` ([`7b1cfc54c`](WebKit/WebKit@7b1cfc54c)). `-1` was 4294967295, so `Array.prototype.slice.call(arguments)` looped 2^32 times, and 2^32 and up were truncated. - `Stopwatch::reset()` clears past run intervals ([`49902180d`](WebKit/WebKit@49902180d)), and moving a `SamplingProfiler::StackTrace` keeps `stopwatchTimestamp` ([`b6c984205`](WebKit/WebKit@b6c984205)). - DFG fixes: a promise resolved with itself was fulfilled with itself in optimized code instead of rejected with a TypeError ([`b45f2446d`](WebKit/WebKit@b45f2446d)), `op_has_private_name` is not compiled as `InById` ([`dd24421c8`](WebKit/WebKit@dd24421c8)). - GC: leftover marks after the marking version wraps ([`e1a28d3cf`](WebKit/WebKit@e1a28d3cf)), a heap's Collector threads stop when it shuts down ([`eaf0efa57`](WebKit/WebKit@eaf0efa57)). ### How did you verify your code works? No local build. The first commit was built by CI ([#131689](https://buildkite.com/bun/bun/builds/131689)): all 13 `build-bun` jobs and the three `verify-baseline` jobs passed, so the aarch64 allowlist stays as it is, and the only test that failed was the bytecode snapshot, on all ten platforms that ran it. With the `bun-linux-x64` binary of that build, and the one of `main` at `a8fcd319` ([#131688](https://buildkite.com/bun/bun/builds/131688), WebKit `5e3554face13`) as the baseline: | | `main` | this PR | | --- | --- | --- | | `webkit-upgrade-f59345e04a.test.ts` | 0 of 5 pass | 5 of 5, also with `BUN_JSC_useJIT=0` | | `bundler_bytecode_portable.test.ts` | | 22 of 22 with the new snapshot, 21 of 22 before | | `source-positions.test.ts` | | 325 of 325 | | `inspector-profiler.test.ts` | | 52 of 52 | The snapshot was written by `--update-snapshots` on Linux x64. All 24 hashes are the ones in the CI logs of Windows x64, Linux aarch64 and Linux x64 ASAN, and the 11 that the annotation shows are the same on all ten platforms. The second commit's Rust was checked with `cargo check -p bun_jsc -p bun_runtime`; its C++ (one signature) is first compiled by CI. Before the first build, by reading: the diff of every JavaScriptCore, WTF and bmalloc header in the range, with each removed or changed name searched for in Bun, and every `JSBigInt::` member Bun uses looked up in the new `JSBigInt.h`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…k.ts` (oven-sh#46641) ### What does this PR do? Moves the stream sinks to bindgen and deletes `src/codegen/generate-jssink.ts`. `src/runtime/webcore/Sink.bind.ts` declares four classes: | Class | What it is | | --- | --- | | `ArrayBufferSink` | `Bun.ArrayBufferSink` | | `FileSink` | `Bun.file(..).writer()`, `Bun.stdout.writer()`, `proc.stdin` | | `NetworkSink` | `s3file.writer()` | | `ReadableSinkController` | the `controller` of a `type: "direct"` stream that a native sink consumes, and what the pump of any other stream writes to | The old generator made a sink class, a constructor and a controller class for each of the seven sinks. Four of the sink classes were never instantiated, and the controllers differed only in which sink they forwarded to. There is now one controller class, as `ReadableStreamDirectController` in `bun-types` and the docs already describe it. It holds an enum of typed pointers to the six sinks, and each operation picks the sink with one `match` in `Sink.rs`. Bindgen parses the arguments: `write(chunk)` is a union of a string, a buffer source and anything else, `flush(wait)` a boolean, and `start(options)` a dictionary of the members that sink reads. `Bun.file(..).writer(options)` uses the same dictionary as `FileSink.start`. The logic that was generated C++ (attach, detach, `end`, `close(error)`, the closed and destination-closed states) is in `Sink.rs`, and `BunStreamSource.cpp` keeps the stream side. The two call each other through member functions of the generated class, which the schema declares, so neither side declares an `extern "C"` function for them. This is what `GeneratedSink.h` has for the controller: ```cpp // Defined in Rust. On exception, a JSValue result is empty. void close(JSC::JSGlobalObject*, JSC::JSValue error); JSC::JSValue end(JSC::JSGlobalObject*); JSC::JSValue write(JSC::JSGlobalObject*, JSC::JSValue data); bool isDetached(); void closeWithError(JSC::JSGlobalObject*, JSC::JSValue error); void startWithHighWaterMark(JSC::JSGlobalObject*, double highWaterMark); JSC::JSValue writeBytes(JSC::JSGlobalObject*, std::span<const uint8_t> bytes); // Defined in hand-written C++, called from Rust. JSC::JSValue assignToStream(JSC::JSGlobalObject*, JSC::JSValue stream); void onClose(JSC::JSGlobalObject*, JSC::JSValue reason, bool sinkClosed); void onReady(JSC::JSGlobalObject*); void setPipe(JSC::JSValue stream, JSC::JSValue done); JSC::JSValue pipeStream(); void clearPipeStream(); void setPipeDone(JSC::JSValue done); JSC::JSValue pipeDone(); JSC::JSValue takePipeDone(); void setFailReason(JSC::JSValue reason); JSC::WriteBarrier<JSC::JSCell> m_source; JSC::WriteBarrier<JSC::JSPromise> m_closePromise; JSC::WriteBarrier<JSC::Unknown> m_failReason; JSC::WriteBarrier<WebCore::JSReadableStream> m_readableStream; JSC::WriteBarrier<WebCore::JSReadableStream> m_pipeStream; JSC::WriteBarrier<JSC::JSPromise> m_pipeDone; ``` The pump now holds a `JSReadableSinkController*` and calls `write`, `end` and `close` directly. Before, it looked them up by name and built an options object for `start`. They are the same functions JS calls, so arguments convert the same way. Also gone: `Sink.h`, `generated_jssink.rs`, twelve lazy members of `ZigGlobalObject`, seven `DOMStructureSlot` entries, six iso subspace slots, the sink block of `headers.h`, `GlobalObject::assignToStream`, `host_fn_static` and `StartTag`. `RequestContext` holds its `HTTPServerWritable` directly, and `JSTransformStream` holds the controller in one `m_nativeSinkController` in place of a cell, a pointer and an id. The sinks were the last Rust code to read a `CallFrame`, so these have no users and are deleted: - `CallFrame::{arguments, argument, arguments_count, this}` and the copy of JSC's register layout that they decoded - the `CallFrame` parameter of `js_thread_of_caller` and `context_of_caller`, which leaves one function of each name - `JSGlobalObject::throw_illegal_constructor`, three `BuiltinName`s, three methods of `TaggedPtrUnion` and the tag bounds of `TypeList` - the `flush()` of four sinks, which only the old trait called #### Bindgen additions | Option | What it does | | --- | --- | | `writable: false` on a class | its native functions are read-only | | `{ constant: 3 }` in `proto` | a read-only integer, like a WebIDL `const` | | `enumerable` on a getter or a constant | overrides the class's, as `configurable` does | | `calledFromCpp: true` on a function of `proto` | also a member function of the generated C++ class: it takes a `JSValue` for each argument and converts it as a call from JS does | | `calledFromCpp: { .. }` on a class | member functions defined in Rust that are not properties; they take and return `JSValue`, `bool`, numbers and `b.byteSpan` as they are | | `calledFromRust: { .. }` on a class | member functions that hand-written C++ defines and Rust calls as `js_Name::x(this_value, ..)` | | `b.byteSpan` | bytes that C++ passes to Rust: `std::span<const uint8_t>`, `&[u8]` | | `internalFields: { done: b.cell("JSC::JSPromise", header) }` | a field for hand-written C++, a `WriteBarrier<JSC::JSPromise>`; fields declared as a list stay `WriteBarrier<Unknown>` | | `b.string.view` | the characters of a string where they are, as a `bun_core::StringView<'a>`: no reference is taken and a slice is not copied | | `.falsyAsNull` on `b.any` | a falsy value is `None` | ### Behaviour A script calls every method of an `ArrayBufferSink`, a `FileSink` and a server's controller with 33 kinds of argument, with no argument, with five kinds of wrong `this`, and after `close()` and `end()`. It also covers each member of the `start` and `writer` options, the constructors and every property descriptor. It ran on main and on this branch; these are the cases that differ. **Wrong on main, fixed here** | Case | main | This PR | | --- | --- | --- | | `Bun.ArrayBufferSink.prototype` | `undefined` | the prototype | | `sink instanceof Bun.ArrayBufferSink` | throws `instanceof called on an object with an invalid prototype property.` | `true` | | `class X extends Bun.ArrayBufferSink {}` | throws `The value of the superclass's prototype property is not an object or null.` | works, and `Reflect.construct` honours `newTarget` | **Different, neither wrong** | Case | main | This PR | | --- | --- | --- | | The controller's label in `console.log`, `Object.prototype.toString` and `heapStats()` | `ReadableHTTPResponseSinkController` and five others | `ReadableSinkController` | | `sinkId` of a controller | a data property of each prototype | an accessor, with the same value and attributes; on the prototype itself it is `undefined` | | A sink's method with a wrong `this` | `Expected FileSink`; no `code` from `close`, `_getFd` and a controller's `end` | `Expected this to be instanceof FileSink, but received ..`, always with `code: "ERR_INVALID_THIS"` | | `FileSink()` without `new` | `Illegal constructor` | `FileSink constructor cannot be invoked without 'new'` | | `start(options)` of an `ArrayBufferSink` | `asUint8Array` counts only if it is an own property | it counts inherited too, like `stream` and `highWaterMark` | | `start(options)` of a controller | reads `path` and `fd` for a `FileSink`, and `highWaterMark` for any other sink | reads all three, and uses the ones main uses; a getter that throws on one it does not use makes `start` throw | | `start({ path, fd })` and `writer({ path, fd })` | `fd` is read only if `path` is absent | both are read, and `path` still wins | | `write()` with no argument | `ERR_MISSING_ARGS`, `write() expects a string, ArrayBufferView, or ArrayBuffer`; a closed sink says that it is closed, and a controller whose client went away returns 0 | `ERR_MISSING_ARGS`, `The "chunk" argument must be specified` (`"data"` on a controller), whatever the state | | `estimateShallowMemoryUsageOf(new Bun.ArrayBufferSink())` | 96 | 80 | | `start(options)` with a wrong `this` | runs the getters of `options`, then throws `ERR_INVALID_THIS` | throws `ERR_INVALID_THIS` first | | `write(new String(..))` on a closed sink | throws before it converts the chunk | converts the chunk, then throws the same error | | A method borrowed across classes | a sink and its controller take each other's `write`, `flush` and `start`, and a controller of another sink does not | a sink's methods work on that sink only, and a controller's on every controller | **Right on main, wrong here**: none. `Sink.rs` discarded the result of detaching a controller. It goes to `dispatch::fold` now, like the same call in `FileSink.rs` and `streams.rs`, and the `jsresult-swallow` inventory loses the entry. ### Cost Release builds of main and of this branch, one pinned core, cycles for each operation. The third column is a second copy of main's binary. | Operation | main | main again | This PR | | --- | --- | --- | --- | | a direct stream to a file | 4594 | 4571 | 4028 | | `Bun.serve` with a stream body | 12487 | 12537 | 11547 | | `await Bun.write(path, new Response(stream))` | 8412 | 8439 | 7521 | | `arrayBufferSink.start({ highWaterMark })` | 243 | 246 | 102 | | `arrayBufferSink.flush()` | 369 | 369 | 321 | | `arrayBufferSink.write("hello world")` | 106 | 107 | 89 | These cost more: | Call | main | main again | This PR | | --- | --- | --- | --- | | `arrayBufferSink.write(uint8Array)` | 76 | 73 | 78 | | `arrayBufferSink.write(arrayBuffer)` | 90 | 88 | 94 | | `fileSink.write(uint8Array)` | 101 | 100 | 111 | | `fileSink.flush()` | 62 | 59 | 69 | | `controller.write(uint8Array)` | 80 | 82 | 102 | | `controller.flush()` | 39 | 40 | 51 | On main `write` and `flush` are each one Rust function that JSC calls and that reads its own argument. Now a generated C++ function checks `this` and converts the argument, and then calls the Rust function. A profile of `controller.write(uint8Array)` shows nothing else: three frames where main has one, and a call back into C++ for the pointer and length of the buffer, which `b.bufferSource` reads when the bytes are used and not when the argument is converted. The other calls of the three classes (`write` of a string to a `FileSink` or a controller, `start()`, `end()`, `ref()`, `unref()`, `_getFd()`, `sinkId`, `new Bun.ArrayBufferSink()`, `Bun.file(..).writer()`) measure within the difference between the two copies of main, or faster. The GC calls Rust when it sweeps a controller. On main the destructor of a detached controller made no call. `JsLocal<BufferSource>::with_bytes` gets the bytes from `Bun__Bindgen__bytesOfBlobOrBuffer`, as a `b.bytes` handle does, in place of `JSC__JSValue__asArrayBuffer`, which looked at the type again and filled a struct. The release binary is 86,016 bytes smaller than main's. ### How did you verify your code works? - `test/js/bun/util/arraybuffersink.test.ts` and `test/js/bun/http/serve-direct-readable-stream.test.ts` each have a new test that fails on main. - `test/internal/bindgen.test.ts` covers each bindgen addition, and each mistake in a schema that one of them reports. `TestMemberFunctions` goes from JS to Rust to C++ (`BindgenTest.cpp`, only in a build with assertions) and back to Rust, and keeps a `JSPromise` in a typed field across a GC. - `test/integration/bun-types/bun-types.test.ts` compares the schema with `Bun.ArrayBufferSink`, `Bun.FileSink`, `Bun.NetworkSink` and `ReadableStreamDirectController`. Its list of known differences gains the members that `bun-types` does not declare (`close`, `_getFd`, `sinkId`, and `ref` and `unref` of an `ArrayBufferSink`), `NetworkSink.stat()`, which `bun-types` declares and no sink has, and the `highWaterMark` of `FileSink.start(options)`, which `bun-types` declares and a `FileSink` does not read, on main as well. - These pass on a debug build: `bindgen`, `arraybuffersink`, `filesink`, `serve-direct-readable-stream`, `direct-readable-stream`, `async-iterator-stream`, `serve-stream-body-error`, `s3-upload-stream-gc`, `s3-stream-error-gc`, `spawn-stdin-destroy`, `spawn-stdin-readable-stream`, `text-encoder-stream`, `compression`, `body-stream`, `fetch-abort-stream-body`, `html-rewriter`, `html-rewriter-end-error`, `node-tls-upgrade`. - `bindgen`, `arraybuffersink`, `filesink`, `serve-direct-readable-stream`, `serve-stream-body-error`, `s3-stream-error-gc`, `compression` and `text-encoder-stream` also pass with `BUN_JSC_validateExceptionChecks=1`. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…ven-sh#46718) - `host.sh` installs the pinned Bun when `/usr/local/bin/bun` reports a different version. - `joinTailnet` returns when `tailscale status` already succeeds, before `install-system-daemon`. - `guest/bake.sh` runs `brew update` before the toolchain script. - The README says the scripts run on the Bun `host.sh` pinned when it last ran on the host.
…#46505) ### Problem - `console.groupCollapsed()` does not start a group. Later lines are not indented, and its `console.groupEnd()` removes the indent of an enclosing `console.group()`. A call with no label writes `undefined`. - Cause: `message_with_type_and_level_` (`src/jsc/ConsoleObject.rs:383`, `:394`) tests `MessageType::StartGroup` only. JSC sends `StartGroupCollapsed`. ### Fix - One `is_start_group` flag covers both types. The `undefined` fallback is removed: a collapsed group with no label was its only caller. The `bun-types` JSDoc is updated. - Correct: Node documents `groupCollapsed` as [an alias of `group`](https://nodejs.org/docs/latest-v26.x/api/console.html#consolegroupcollapsed), and the [WHATWG spec](https://console.spec.whatwg.org/#groupcollapsed) pushes both onto one group stack. - Verified: `test/js/web/console/console-log.test.ts`, `test/js/bun/test/dots.test.ts` (new tests fail on bun 1.4.3-canary). Also ran: console test directories, bun-types test. - Self-reviewed: 15 concerns raised, 15 addressed. Notes names two follow-ups that are not in this PR. ### Background - JSC passes each console call, with a `MessageType`, to `Bun::ConsoleObject::messageWithTypeAndLevel`. It forwards the message to the inspector, then to the Rust printer, which keeps the group depth in `default_indent`. - Considered a JS alias of `group` and a type mapping in the C++ client. The alias hides the collapsed type from the inspector, which can fold a group. The mapping splits one printer rule across two files. <details><summary>Notes</summary> Repro: ```js console.group("outer"); console.groupCollapsed("inner"); console.log("x"); console.groupEnd(); console.log("still in outer"); console.groupEnd(); console.log("after"); ``` | bun 1.4.3-canary (367d939) | this PR | Node v26.3.0 | |---|---|---| | `outer` | `outer` | `outer` | | `··inner` | `··inner` | `··inner` | | `··x` | `····x` | `····x` | | `still·in·outer` | `··still·in·outer` | `··still·in·outer` | | `after` | `after` | `after` | (`·` is a space.) **History** - There is no user report for this bug. A read of the code found it. It dates from oven-sh#15026, which added `console.group` for oven-sh#8099 and handled `StartGroup` only. - This is a refile of oven-sh#34237, which had the same two conditions. The stale action closed it after 90 days with no human review. - oven-sh#39592 moves `node:worker_threads` workers to the native console, and its branch has the `StartGroup`-only checks. Those workers print Node's output for `groupCollapsed` today. They keep it if this change lands first, or if oven-sh#39592 is rebased over it. The two diffs do not overlap. - oven-sh#42163 mirrors a page's `groupCollapsed` through `ConsoleClient::group()`. It does not need this change. **Scope** - The same code path serves `console.warn` and `console.error` inside a collapsed group. It also serves the console of a web `Worker`. Both are corrected. - Not affected: `new console.Console()` and `node:worker_threads` workers. They use the JS `Console` class, where `groupCollapsed` is already an alias of `group` (`src/js/builtins/ConsoleObject.ts:780`). Node does the same ([`constructor.js:681`](https://github.com/nodejs/node/blob/v26.3.0/lib/internal/console/constructor.js#L681)). - An outside caller: `Console.withGroup({ collapsed: true })` of `effect` 3.22.3 calls `console.groupCollapsed` on the global console (`src/internal/defaultServices/console.ts:57`). Nested in another group, it loses the outer indent on main and prints Node's output with this change. - Not changed: some output stays at column 0 inside a group of either kind. That is `console.count`, `console.timeLog`, `console.timeEnd`, `console.table`, and `console.assert` with no message. The later lines of a multi-line string stay there too. Related: oven-sh#30017, oven-sh#36470. The new JSDoc names only methods that do indent. **The removed fallback** - It was `else if message_type != MessageType::Trace { writer.write_all(b"undefined\n") }`, with no flush and no indent. On main, the text for a collapsed group with no label waits in the buffer. It shows up with the next `console.*` line on stdout, or not at all. - To reach it, a message needs no arguments, a type other than `Log` or `Trace`, and no earlier return. JSC drops `console.dir()`, `console.dirxml()` and `console.table()` with no arguments (`ArgumentRequirement::Yes` in `ConsoleClient.cpp`). `EndGroup`, `Clear`, `Assert` with no arguments, and now both group types return earlier. JSC sends no other type through this function. - Checked with a run of 19 console methods with no arguments. Main prints `undefined` for `groupCollapsed` only. This PR prints it for none. - Without the fallback, the early return for a group with no label is visible only under `bun test`. There it skips the before-print hook of the reporter. On main, a collapsed group with no label breaks the line of dots of `bun test --dots`. The new case in `dots.test.ts` pins the return. With the return put back to `StartGroup` only, that case fails and the case in `console-log.test.ts` passes. **Follow-ups that are not in this PR** - Node prints `undefined` for `console.dir()` and `console.table()` with no arguments, and an empty line for `console.dirxml()`. Bun prints nothing for the three on the global console. - `bun test --isolate` does not reset the group depth, the `console.count` totals or the `console.time` timers between test files. **Cost** - On a reduced copy of the two conditions (rustc 1.100.0-nightly, `opt-level=3`), `cmp; sete` becomes `add; cmp; setb`. No allocation, write or syscall is added. **Suites run with the debug build** - `test/js/web/console/`, `test/js/node/console/`, `test/js/bun/console/`, `test/js/bun/test/dots.test.ts`, `test/js/bun/test/only-failures.test.ts`: 118 pass, 0 fail. - `test-console-group.js`, `test-console-methods.js` and `test-console-with-frozen-intrinsics.js` from Node's suite: exit 0. - `test/integration/bun-types/bun-types.test.ts`: 24 pass. </details> ### Downsides - Output changes for callers of `console.groupCollapsed`: later lines gain two spaces for each open group. Snapshots of that output change. - Under `bun test`, a collapsed group that one file leaves open now indents the output of later files. An open `console.group()` does this today. - One more instruction per console message (a range check replaces an equality check).
…46439) ### What does this PR do? Four things for `bun lint`. #### `bun lint` is a command in release builds This reverts oven-sh#46438, which kept `bun lint` out of release builds. Bun v1.4.4 was released with it, so the next release is the first one that has `bun lint`. | in a release build | before | after | |---|---|---| | `bun lint` | the project's own `lint` script, file or executable, else `error: Script not found "lint"` | the linter, unless the project has its own `lint` | | `bun --help` | does not list `lint` | lists `lint` | | README from `bun init` | "Run `bun fmt` to format the code." | "Run `bun lint` to find problems in the code and `bun fmt` to format it." | | test runner | skips `test/cli/lint/` and the cases of `init.test.ts`, `bunfig-errors.test.ts` and `glob/match.test.ts` that run `bun lint` | runs them | | order file generator | does not trace `bun lint` | traces it | `docs/lint/index.mdx` loses the note that the command is in canary builds only. Nothing changes in a canary build. #### `require.resolve()` in an installed plugin adds only Node's extensions eslint-plugin-import-x looks for its resolver named "node" with `require.resolve(<directory of the nearest package.json> + "/node")` in a try / catch. In Node.js that throws, and the plugin goes on with its built-in resolver. Bun also adds `.ts`, so in a package that has a `node/index.ts` the plugin loaded that file as its resolver, which ran the project's code, and every file of the package got a false error: ``` 1:1 error Resolve error: node with invalid interface loaded as resolver import/order ``` | `require.resolve(path.resolve("node"))` in `node_modules/eslint-plugin-demo/index.js`, in a project with `node/index.ts` | | |---|---| | ESLint 10.12.0 on Node.js 26 | throws `MODULE_NOT_FOUND` | | `bun lint` before | `<project>/node/index.ts` | | `bun lint` now | throws `MODULE_NOT_FOUND` | ESLint runs plugins in Node.js. So `require()` and `require.resolve()` in a file under `node_modules` now add `.js`, `.json` and `.node` where they added Bun's default list, in the threads that run the rules and in the process that reads a configuration for `bun lint` and `bun fmt`. An extension that a package adds to `require.extensions` still counts. Nothing else changes: not `bun run`, not `import`, not the project's own files. In directus, 103 such errors are gone, and the output is ESLint's line by line. The output for Leaflet, jotai and vite is the same as before. #### `-A`, `-W` and `-D` reach a rule that's configured under two names With an `.oxlintrc.json`, `no-array-constructor`, `eslint/no-array-constructor` and `typescript/no-array-constructor` are one rule. If the file and the files it extends used two of these names, the flags missed the rule, so a run with `-D` passed that should fail. ```jsonc // .oxlintrc.json { "extends": ["./base.json"] } // base.json { "rules": { "typescript/no-array-constructor": "warn", "no-array-constructor": "warn" } } ``` | `export const a = new Array(1, 2, 3);` | oxlint 1.87.0 | `bun lint` before | `bun lint` now | |---|---|---|---| | `-D no-array-constructor` | error, exit 1 | warning, exit 0 | error, exit 1 | | `-A no-array-constructor` | nothing, exit 0 | warning, exit 0 | nothing, exit 0 | | both names set to `"error"`, `-W no-array-constructor` | warning, exit 0 | error, exit 1 | warning, exit 0 | | both names set to `"error"`, `-A no-array-constructor` | nothing, exit 0 | error, exit 1 | nothing, exit 0 | The same happened with one name in the file and the other in a file it extends, with the two names in two extended files, and with `-A all` or a category in place of the rule. With two extended files that give the two names different severities, a flag for another rule, like `-D eqeqeq`, changed which of the two applied. oxlint merges files by the names as they're written, and applies the flags to the rule afterwards. We copy the inherited rules into the file's own `rules` and change them there, and that copy skipped a rule that was already there under another name. Now each written name is copied. | compared with oxlint 1.87.0 | results | differed before | differ now | |---|---|---|---| | 194 configurations with two or three names of one rule, 13 sets of flags each | 2,522 | 443 | 0 | | 210 configurations where the two names have different options, 7 sets of flags each | 1,470 | 344 | 0 | Without flags, none differed before. #### Shorter hints Shortens the hints of `bun lint`: the `note:` line that `-f pretty` and `-f agent` print under a report whose message doesn't say why or what to do. They used more words than they needed, and often began with a vague "It is". ``` 1 | export function price(items: number[]) { 2 | let total = 0; ^ warn: The value assigned to 'total' is not used in subsequent statements. no-useless-assignment at src/price.ts:2:7 note: Reassigned on line 3 before it's read. ``` | rule | before | after | |---|---|---| | `no-useless-assignment` | It is overwritten on line 3 before it is read. | Reassigned on line 3 before it's read. | | `react-hooks/rules-of-hooks` | The return on line 4 can end the render before it is called. | The return on line 4 can skip this hook. | | `react-hooks/rules-of-hooks` | Whether it is called depends on the condition on line 9. | Depends on the condition on line 9. | | `no-useless-catch` | It only throws the error again. Without try and catch, the code does the same. | It only rethrows. Remove the try and catch. | | `no-dupe-keys` | Only the last one counts. The first is on line 2. | The last one wins. The first is on line 2. | | `no-compare-neg-zero` | -0 and 0 compare as equal. Use Object.is(x, -0) to tell them apart. | -0 === 0 is true. Use Object.is(x, -0). | | `no-compare-neg-zero`, `x < -0` | -0 and 0 compare as equal. Use Object.is(x, -0) to tell them apart. | -0 compares like 0. Write 0. | | `no-empty-character-class` | [] in a regular expression matches nothing. | [] matches nothing. | | `no-fallthrough` | Add break, or the comment // falls through if this is meant. | Add break, or a // falls through comment. | | `no-this-alias` | Use an arrow function instead. It has the this of the code around it. | Use an arrow function. It keeps the outer this. | | `no-explicit-any` | unknown is the safe choice where the type is not known. | Use unknown if the type isn't known. | | `no-unused-vars` | Remove it, or start its name with _ to keep it. | Remove it, or prefix it with _. | | `no-unused-vars` | A catch needs no parameter. Remove it and its parentheses. | The parameter is optional. Write catch { }. | | `no-const-assign` | It is declared with const on line 1. Use let if it has to change. | Declared with const on line 1. Use let if it changes. | | `no-octal` | The leading 0 makes it octal, so 071 is 57. Write 0o71 to make that clear. | The leading 0 makes it octal: 071 is 57. Write 0o71. | 51 hints of 36 rules change this way. Two lines are now listed as "lines 5 and 7", not "line 5 and line 7". The hint of `no-compare-neg-zero` now follows the operator: `Object.is` was the wrong advice for `<`, `<=`, `>` and `>=`, and `!=` and `!==` need `!Object.is`. Nothing else changes: the messages themselves are upstream's, and which reports get a hint stays the same. ### How did you verify your code works? The revert: it applies without conflicts, each of its 14 files is byte for byte what it was before oven-sh#46438, and nothing that `main` got since uses what it removes (`isCanary`, `IS_ENABLED`, the `STABLE` modifier). PR CI builds are canary, so CI does not cover the release column. No release build was made of it. The plugin fix: a new test in `test/cli/lint/js-plugins.test.ts` with a stand-in plugin, at the time the configuration is read and at the time a rule runs. It fails without the fix, and what it expects is what ESLint 10.12.0 prints. Not run on macOS or Windows. The hints: `test/cli/lint/hints.test.ts` has a case for each hint and expects the new text. It passes, as do `output.test.ts` and `git.test.ts`, which also quote a hint. Two tests that check that a format prints no hint looked for a word of the old text, which would have passed whatever was printed. They look for the new text now.
… files without an ESLint or oxlint config (oven-sh#46980) Fixes oven-sh#46971 ### Problem - With plugins loaded from `bunfig.toml` alone, a JS plugin rule sees `node.start` and `node.end` in `.js` files but not in `.ts` files. There, nodes, tokens, and comments have only `range` and `loc`. A rule that compares `comment.end <= node.start` evaluates `undefined <= undefined` and silently misfires. - `LintNodeState::kindFor` (`src/jsc/bindings/LintNodes.h`) picked typescript-estree's layout for every non-espree, non-oxlint file. With an ESLint config that is right. Without one there is no parser to match, and the run mixes two shapes. ### Fix - One function, `estree::has_start_end(file)`, says whether a file's nodes have `start` and `end`: espree's and oxlint's do, and typescript-estree's do when `LanguageOptions::has_no_foreign_config` is set. The selector evaluator (`selector/value.rs`) and the plugin host (`host.rs`, as a new flag bit) both ask it. - A fifth node layout in `src/lint/js_plugin/layouts.rs`, `TypeScriptWithStartEnd`: typescript-estree's properties with `start` and `end` right before `range`. `kindFor` picks it when `LintHost::hasStartEnd()` is set for a file that is neither espree's nor oxlint's. - Tokens and comments of those files get the same two properties through two new `LintTokenShape` kinds. The nodes that `worker/ast.js` builds for typescript-estree's deprecated properties follow the node they are built for. - The ESLint-config and oxlint-config shapes are unchanged. Their oracle tests still pass. - Verified: `test/cli/lint/js-plugins.test.ts`, two new tests (the tree oracle under `bunfig.toml`, and keys and offsets of a node, a type annotation, both comment kinds, a token, and a regex token in `.ts` and `.js`, plus a `[start=13][end>13]` selector). Also the whole file (148 pass), `js-parsers.test.ts`, `bunfig.test.ts`, `rule-throws.test.ts`. ### Background - `layouts.rs` is the one table that says which properties a node of each type has, in which order, per parser. Rust sends it to the plugin worker once. `LintNodes.cpp` builds a JSC Structure per node type from it, so every node of a file passes through it. `OxlintJavaScript` is an earlier variant of the same kind. - `has_no_foreign_config` is the linter's existing flag for "no ESLint or oxlint config, so no installed tool's behaviour has to be matched". The linter already uses it to clamp report locations and to remove whole lines in fixes. - Considered adding `start` and `end` to the TypeScript layout for everyone: plugins that spread nodes or compare keys would see a shape that typescript-estree does not produce, and the ESLint oracle test pins that shape. Considered oxlint's layout for every file under `bunfig.toml`: it changes `.js` files too (oxlint field order, `null` for absent fields, inherited `loc`, oxlint scopes and tokens), while the documented use case is ESLint plugins from npm. - `.oxlintrc.json` is not a substitute: it flips `has_no_foreign_config` and the other behaviours keyed on it. ### Downsides - Each plugin worker thread loads the layouts table once. It grows from 71,530 to 98,250 bytes (+26,720). Templates and Structures are made per node type on first use, so files that never hit the new layout make none. - A plugin that detects the parser by `"start" in node` sees typescript-estree's nodes with espree's offsets under `bunfig.toml` alone. No such check was found in the test suite or in the plugins that the docs name. - What stays typescript-estree's in this mode: an absent field is `undefined`, not `null` as in oxlint. A rule written for oxlint that checks `=== null` still differs. The docs bullet says so. <details><summary>Notes</summary> - Self-reviewed: 4 concerns raised, 4 addressed (docs wording, the synthesized nodes in `worker/ast.js`, a regex token shape without a test, the tree oracle run in the new mode). Review on the PR found two more, both addressed: selectors with `[start]` or `[end]` evaluated natively against the old rule, now through the same `has_start_end`; the synthesized nodes read a per-file global, now they look at the node they extend. - The tree oracle under `bunfig.toml` matches the ESLint oracle with `start: a, end: b,` inserted before every `range: [a, b]` of the non-`.js` files, including the nodes behind `TSMappedType.typeParameter` and `TSImportType.argument`. The only other difference is `sourceType`, which Bun derives from the file where ESLint's config sets it, so the test masks it. - Layout kinds in C++ now have `isTypeScript` and `isOxlint` predicates in place of `==` and `>=` comparisons, so a new kind at the end cannot fall into the wrong branch. - The `visitorKeys` numbering (0 typescript-estree, 1 espree, 2 oxlint) is unchanged. The new kind maps to typescript-estree's keys. </details>
…#46486) ### What does this PR do? Follow-up to oven-sh#46468. It fixes the three cases that the reviews of that pull request left open, and three more that the reviews of this one found. All are in `find_in_project`, on macOS and Linux. A native dependency is a package in `nativeDependencies` of the project's package.json, or in the built-in list when the project has no such field. For those, `bun install` links `node_modules/.bin/<name>` into the platform package: a dependency with an `os` and a `cpu`. | # | Case | Own bin file | Before | After | |---|---|---|---|---| | 1 | A folder above the project has the name of an optional dependency that comes before the platform package | `0644` | `is not executable`, exit code 126 | platform binary | | 1 | the same | `0755` | stub | platform binary | | 2 | The package is not a native dependency. `.bin/<name>` is the bin of one of its optional dependencies, which has an `os` and a `cpu` | `0755` | bin of the dependency | own file | | 3 | `.bin/<name>` is the bin of an optional dependency without `os` and `cpu` | missing or not executable | `is not executable`, exit code 126 | `.bin/<name>`, as before oven-sh#46468 | | 4 | The platform package of a native dependency is in `dependencies` | `0644` | `is not executable`, exit code 126 | platform binary | With case 3 alone, `.bin/<name>` of an unrelated package would also run in two layouts. Both keep the error of `main`: | # | Layout | Own bin file | Result | |---|---|---|---| | 5 | A folder above the project has the name of an optional dependency, and a package.json | not executable | `is not executable`, exit code 126 | | 6 | `.bin/<name>` is in `@scope/first`, and the package has the optional dependency `first` | not executable | `is not executable`, exit code 126 | | # | Change | |---|---| | 1 | `dependencies_with` returns every dependency whose name is in the real path of the link, not only the first | | 2 | `is_in_platform_package` also asks `List::is_native_dependency`. That reads the project's package.json with the parser of the installer. It looks up the `name` in the package's own package.json, so an alias works | | 3 | `find_without_executable_file` accepts a link into any optional dependency again | | 4 | `is_in_platform_package` looks in `dependencies` and `optionalDependencies`. The installer has no filter on the kind | | 5 | `dependencies_with` skips a folder that the package itself is in | | 6 | `dependencies_with` skips a folder whose parent begins with `@` | ### How did you verify your code works? - The tests in `bunx.test.ts` whose names begin with `after an install, a package whose optional dependency`, `a package that cannot run` and `the native dependency`. I ran each on a build without its fix first, and each failed there. All pass now. - `after an install, esbuild in a project with nativeDependencies <undefined | []>` covers the built-in list. It passed before too. - The rest of `bunx.test.ts` and `bun-install-native-binlink.test.ts` pass on macOS arm64, but for `should handle postinstall scripts correctly with symlinked bunx`. My machine does not let a copy of bun run from the temp folder. That test asks for `esbuild@latest`, and a request with a version does not reach this code. Linux is left to CI.
Fixes oven-sh#47006 ### Problem - `bun install` writes `[0.02ms] ".env"` to stderr whenever a `.env` file exists. `--silent` and `--quiet` do not remove it. Every install-family command (`add`, `remove`, `update`, `bunx`, ...) prints it. - `PackageManager::init` allocates its dotenv loader with `Loader::init()` (`src/install/PackageManager.rs:2384`) and never sets `quiet`. `Loader::load` (`src/dotenv/env_loader.rs:660`) prints the line unless `quiet` is set. ### Fix - The loader gets a `quiet_loaded_files` flag that hides only the `[Nms] ".env"` line. The `quiet` flag keeps its meaning: it also hides the `error loading .env file` messages (`env_loader.rs:816`, `:835`, `:879`), which `bun run` relies on. - Install sets `env.quiet_loaded_files = !cli.log_level.is_verbose()` right after the loader is allocated. This is the gate install already uses for its other `print_elapsed` timing traces. The line shows with `--verbose` (and under `RUNNER_DEBUG`, which promotes the level). - The `.env` files are still loaded, and an unreadable `.env` still prints `EACCES error loading .env file` at every level, as before. - Verified: `test/cli/install/bun-install-env-file.test.ts` (46 pass, 1 skip as root. The three new "not printed" cases fail on bun 1.4.3). Also `test/cli/run/env.test.ts` and `test/cli/run/log-test.test.ts`. ### Background - `bun run` and the transpiler already set `quiet` on their loaders: `run_command.rs:660` forces it, `transpiler.rs:1175` derives it from the log level (default `Warn`, so hidden). Install was the only `Loader::load` caller that left the line on. - Considered setting `quiet` on the install loader, as `bun run` does. That also hides the read error for a `.env` that carries the registry token, so a failed auth would have no explanation. The separate flag keeps that message. - Considered flipping the `Loader::init()` default to hidden. It changes nothing for any live loader today (every other `init()` user calls only `load_process`, and the transpiler overwrites the field) and still needs the install line. ### Downsides - A user who read the `[Nms] ".env"` line from the default `bun install` output must pass `--verbose` now. stderr lines for `bun install` with a `.env` in an empty project: 2 to 1 (default), 1 to 0 (`--silent`), 3 to 3 (`--verbose`). - Cost for other callers: one `bool` per `Loader` and one extra compare per `load`, no allocation or syscall. Binary size and instruction deltas were not measured: `bloaty`, `perf` and `strace` are not installed in this environment. <details><summary>Notes</summary> - Repro: `package.json` + `.env` (`FOO=bar`), then `bun install --silent`. Before: stderr is `[0.18ms] ".env"`. After: stderr is empty. - `cli.log_level` is the value `Options::load` later copies into `options.log_level` (`PackageManagerOptions.rs:1143`), so the bunfig and `.npmrc` paths cannot diverge from it. - A second `PackageManager::init` in one process (the `link` retry) reuses the loader and assigns the flag again. - The EACCES test is skipped as root. Checked by hand as `nobody` with a mode 000 `.env`: default and `--silent` both print `EACCES error loading .env file`, as on main. - The first revision set `quiet` on the install loader. Review pointed out that this also hid the read errors, so the loader got the separate flag. - The tests that assert the line is absent after `--no-env-file` or `env = false` pass `--verbose` now, so the assertion can still fail. - Debug binary `size` (PR build): text 643722302, data 65837697, bss 12169944. </details> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…r `Source`, `Path`, `ImportRecord` and resolve results (oven-sh#46834) ### What does this PR do? Fixes a heap-use-after-free in `BuildMessage.position.file`, by removing the false `'static` lifetimes that let it compile. `Source`, `Path`, `ImportRecord`, the bundler's graph types and the resolver's results now carry the lifetime of what they borrow, so the compiler checks this whole area instead of comments. #### The bug ```js const url = URL.createObjectURL(new File(["["], "bad.yaml")); const wide = ("Ā" + url).slice(1); // any specifier that is not 8-bit ASCII await import(wide).catch(e => console.log(e.position.file)); // reads freed memory ``` `Location::init_or_null_impl` stored `Cow::Borrowed(source.path.text)` in a `Cow<'static, [u8]>`. That compiled because `Source.path` was a `Path<'static>`. For a virtual source the path is really a view of `specifier.to_utf8()`, which `Bun__transpileFile` / `Bun__transpileVirtualModule` free when they return, while `process_fetch_log` moves the message into a `BuildMessage` that JS keeps. There are three ways in: a `blob:` URL of an in-memory `File`, `/bun-vfs$$/node_modules/<mod>/x.json5`, and a plugin's `onLoad` answer. No script has to read `.position`: printing an uncaught error reads `location.file` too. It reproduces on 1.4.4 release builds, where the first 8 bytes of the string come back as allocator metadata. #### The cause is a class, not a line `Source`, `ImportRecord`, `Chunk`, `ParseTask` and the resolver's `Result` all said `'static` for bytes they only borrow, and **safe** functions produced those `'static`s from any `&[u8]` (`IntoStr::into_str`, `path_as_static`, `LinkerContext::get_source`, `scoped_action`, the resolver's entry points). The borrow checker could see none of it, so correctness lived in comments, defensive copies and process-lifetime interners. #### What changes **Diagnostics own their bytes, by type.** `Location { file, namespace: Box<[u8]>, line_text: Option<Box<[u8]>> }`; `Clone` is derived on `Location` / `Data` / `Msg`. A `Log` may outlive every `Source` it reports on. This is what fixes the reported bug: `let file = Box::from(source.path.text)` in `init_or_null_impl`. With that, the machinery for deep-copying logs at every hand-over has no job left: `Log::append_to` moves the messages, and `clone_line_text`, `Log::dupe` / `owned_strings`, `clone_with_builder`, `count`, the `StringBuilder` stub and the three `*_with_recycled` copies are deleted. **Everything else borrows, with the real lifetime.** Lifetimes are erased at compile time: no reference counts, no new fields, no size change. | type | now | | --- | --- | | `bun_ast::Source` | `Source<'a>` (`path: Path<'a>`, `contents` / `identifier_name: Cow<'a, [u8]>`) | | `bun_ast::ImportRecord` | `ImportRecord<'a>` | | bundler | `InputFile<'a>`, `Chunk<'a>`, `CssImportOrder<'a>`, `ParseTask<'a>`, `parse_task::Result<'a>`, `ServerComponentParseTask<'a>`, `ResolveQueue<'a>` | | resolver | `Result<'r>`, `PathPair<'r>`, `ResultUnion<'r>`; `resolve<'r>(.., import_path: &'r [u8], ..) -> Result<'r>` | | module loader | `TranspileExtra<'a>`, `ResolveFunctionResult<'r>`, `ParseOptions<'a, 'b>`, `ParseResult<'a>` | | parser | `ScanPassResult<'a>` | | crash handler | `Action<'a>`, `ActionGuard<'a>` | - `Transpiler::parse<'p>` has its own lifetime instead of the `Transpiler`'s. The VM's is a `Transpiler<'static>`, which is why the module loader had to erase every path and source it parsed. - In the bundler, what lives in the graph is `'a`, the arena lifetime `Graph<'a>.heap` already had. A worker thread sees a `ParseTask<'static>`; that one erasure is in `task_callback` / `io_task_callback`, with its contract stated there. - `type Str = &'static [u8]`, `trait IntoStr`, `path_as_static` (16 callers), `unsafe fn Path::into_static` and both `import_path: &'static [u8] = unsafe { .. }` in the resolver (with their `TODO: thread an explicit lifetime through Result`) are deleted. `Path::dupe_alloc` is safe code now. The one holder that has to erase a whole `Source` uses `unsafe fn Source::into_static`. - `scoped_action!` binds its guard to a hygienic local (the `pin!` trick), so the guard cannot be leaked and the thread-local cannot outlive what the action borrows. Safe, same cost. - The scan pass takes the `ScanPassResult` containers by value and gives them back, instead of `&'a mut T<'a>`. #### What the compiler found once the types were true - `PackageInstall::get_installed_package_json_source` returned a `Source` whose path pointed into a local `spill` buffer (E0515). Latent: its log is dropped unread, so nothing dereferenced the path. Now the function returns the contents and the caller names the source. - The module loader could move a virtual source into the async module queue, which outlives it. (That branch cannot run today: nothing fills `pending_imports`. It now refuses a virtual source.) - `bun link` borrowed `manager.original_package_json_path` across a `&mut manager` call. - `bun run` and `Transpiler::resolve_entry_point` returned a resolve result made from a local `./`-prefixed copy of the name. Such a result does not borrow the name in practice, but nothing said so. `resolve_entry_point` now takes the buffer from its caller. - For a framework built-in module, `enqueue_file_from_dev_server_incremental_graph_invalidation` would put a path borrowing its `path_slice` argument into the graph: the case `enqueue_entry_item` had a second copy and a 10-line comment for. #### What existed only to back a fake `'static`, and is gone - `VirtualMachine.resolved_path_dups`: a heap copy of every `blob:`, `macro:`, node-fallback and `[eval]` specifier ever resolved, kept until the VM is destroyed. The one consumer copies the result into a JS string immediately. - `intern_transpile_path` on every synchronous transpile: each distinct path went into the process-lifetime `FilenameStore` and a thread-local map, for the sake of the async queue. It now runs in that branch only. - The second `dupe_alloc` of every entry point, the `*result.path() = path` write-backs, the copy of `tmp_source.path` in `run_resolver`, and the `free_list` parking of plugin `onResolve` results (that arm is now safe code). - A deep copy of every message each time a log was handed from a task to its owner. #### Cost Nothing is added to a hot path. Two places copy where they did not: - A located diagnostic allocates `file` and `namespace` when it is made. Before, it borrowed them and was deep-copied (text, file, namespace, line text, notes) when its log was handed over; now it is moved. - `path_with_pretty_initialized` copies a `node`-namespace path into the bundle's arena instead of returning its argument, and a plugin's external path is copied into the arena instead of keeping its two boxes alive. I have not benchmarked this and make no performance claim. #### Still erased, now visibly Each of these is an explicit `unsafe` with its contract next to it: the two task callbacks; `MapEntry.source`, `PackageJSON.source`, `ParseResult.source_contents_backing` and `eval_source` (self-referential or owner-beside-borrower); the async module branch; `BuiltInModule::Import` in `resolve_with_framework`; `ImportRule.url` at the CSS boundary. Out of scope and unchanged: the CSS crate (`arena: &'static Bump` by convention), AST strings (`StoreStr`, `E::String`), `LinkerOptions`, `Chunk.unique_key`. | in `src/**/*.rs` | main | this PR | | --- | --- | --- | | `.into_static()` calls | 7 | 1 | | uses of the safe erasers `path_as_static` / `into_str` | 26 | 0 | | `&'static [u8] = unsafe { .. }` | 37 | 30 | | `detach_lifetime(..)` | 92 | 84 | | `unsafe` blocks, fns and impls | 12,886 | 12,861 | #### Behaviour changes - `position.file` and `position.namespace` are correct in the cases above. No other message changes (see the probes below). - `Log::append_to` empties the source log; every caller dropped or reset it afterwards. ### How did you verify your code works? - New test in `test/js/bun/resolve/build-error.test.ts`, five cases over the three ways in (`import()` and `require()`). It fails on 1.4.4 (`USE_SYSTEM_BUN=1`, wrong bytes) and on a debug + ASAN build of main (heap-use-after-free), and passes here. - New unit test `location_tests::a_location_outlives_the_path_of_its_source` in `bun_ast`, which drops the source and its buffers before reading the locations. Passes under Miri (`bun run rust:miri -p bun_ast`). - Two differential probes, main (debug + ASAN) against this branch, byte-for-byte identical: - 3,399 `BuildMessage` / `ResolveMessage` objects (every field, `toJSON`, `String`, `Bun.inspect`, notes) from `import()`, `require()`, `blob:` URLs, `Bun.build` and `Bun.Transpiler`, over broken JS, TS, a macro import, a missing import, YAML, TOML, JSON, JSON5 and JSONC, leaving out the cases that are the bug; - 189 `Bun.build` runs: 3 targets × {no plugin, a declining `onResolve`, answering plugins, both} × entry points given as paths, bare names, packages, `data:` URLs and builtins, in-memory `files`, CSS, HTML, externals, dual packages; outputs, hashes, chunk names and logs. - On a debug + ASAN build these files pass with no failure and no ASAN report: - bundler: `bundler_{plugin,plugin_chain,files,loader,html,html_server,edgecase,browser,naming,splitting,barrel,defer,allow_unresolved,regressions,cjs2esm,bun}`, `metafile`, `html-import-manifest`, `plugin-error-nested-throw`, `plugin-sync-exception-fallback`, `css/css-modules` - runtime: `plugins`, `transpiler.test.js` (covers `scan` / `scanImports`), `macro-test`, `runtime-transpiler`, `inspect-error`, `worker_blob`, `run-eval`, `run-cjs`, `run-unicode`, `run-extensionless`, and in `js/bun/resolve`: `build-error`, `resolve`, `resolve-error`, `resolve-ts`, `import-meta`, `import-meta-resolve`, `import-query`, `import-custom-condition`, `non-english-import`, `require`, `bun-main-entry-point`, `load-same-js-file-a-lot`, `tsconfig-extends-leak`, both `resolve-autoinstall-*` - parsers: `json5`, `jsonc`, `toml`, `yaml`, `css-loader`, `invalid-utf8-column` - install: `bun-link`, `bun-pm`, `bun-pm-pkg`, `bun-pm-version`, `bun-workspaces`, `bun-workspaces-self-contained`, `bun-patch` - dev server: `bake/dev/{css,esm,html,plugins,hot,sourcemap,incremental-graph-edge-deletion}`, `dev-and-prod`, `serve-plugins-dev-server`, `framework-router`, `bun-serve-html`, `bun-serve-html-entry` About half of them (the bundler, plugin, transpiler, resolve and dev server ones, `bun-link`, `bun-pm-pkg`) were run again on the final commit; the rest ran one review round earlier. - These fail on my machine in the same way on a debug build of main, so they say nothing either way here and I am relying on CI for them: `bundler/cli.test.ts` (3, `--compile` of a debug binary takes 16 s), `native-plugin` (its build hook times out, so the native plugin's log path was **not** exercised locally), `worker-entry-point` (1), `worker.test.ts` (1 to 2), `run-crash-handler` (1, a symbol name), `bake/dev/bundle` (1, a proxy variable of my environment), and the memory tests `require-cache` (7) and `inspect-error-leak` (1), which time out on a debug build. `bake/dev/production` fails 3 to 9 tests at the 5 s default on both builds and passes 21 / 21 on both with `--timeout 90000` (64 s here, 67 s on main). - `cargo check --workspace --all-targets`, `cargo clippy --workspace --no-deps` and `cargo fmt --check` are clean. `bun run rust:check-all` passes for `x86_64-pc-windows-msvc`, `aarch64-apple-darwin`, `x86_64-unknown-linux-musl`, `x86_64-unknown-freebsd` and `aarch64-linux-android`; I did not run the other triples. Miri passes for the three crates of its CI set that this touches (`bun_ast`, `bun_paths`, `bun_bundler`). --- _Generated by [Claude Code](https://claude.ai/code/session_01GZEGTqHnmApKDDg6554YEM)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ter-free) (oven-sh#46825) ### What does this PR do? Printing an Error could read a source-map file name that had just been freed, and put the stale bytes in the printed stack. This fixes that bug and makes the mistake behind it a compile error. **The bug.** For a stack frame whose URL is not an absolute path, `Lookup::display_source_url_if_needed` returned a string that borrowed the map's own `sources` entry. The printer reads the Error's own `cause` between remapping the frames and printing them. A `cause` getter that loads the same file again replaces the map, which frees its names. `require(URL.createObjectURL(Bun.file("rel.js")))` gives such a URL. - AddressSanitizer build: `heap-use-after-free` under `print_stack_trace`. - Release 1.4.4: the frame prints as `(\^@\^VnO�\^B\^@:1:1)` instead of `(orig.ts:1:1)`. - `Bun.inspect`, `console.log`, `console.error` and an uncaught throw are affected. **The root cause.** `bun_core::String` is the owned type, but `String::borrow_utf8`, `from_bytes` and `ascii` were safe functions that took a `&[u8]` and dropped its lifetime. `StringView<'a>` already existed for borrowing. - Those three constructors are removed. Borrowing goes through `StringView<'a>`, which derefs to `&String`. - `String::clone` copies the bytes a view borrows. It was a bitwise copy, so `view.clone()` and `thread_isolated_copy()` returned a dangling string. - `create_static_external*` take `&'static`. `create_external` is `unsafe`. Both lose `is_latin1`: `false` read past the end of the slice, and every caller passed `true`. - New: `StringOrView<'a>`, a `String` field that may hold borrowed `'a` bytes instead of a ref. It is what a C++ `BunString` slot really is. It derefs to `&String`, drops the ref if it has one, and has no `DerefMut`. - New: `String::from_static_bytes`, `StringView::ascii`, `StringView::as_strings`. **What the compiler then flagged** (112 call sites; most became a `StringView` with no other change): - `ZigException`, `ZigStackTrace`, `ZigStackFrame` and `Holder` get a lifetime, and their strings are `StringOrView<'a>`. C++ fills them with refs, the dev server fills them with slices of the request body, and `remap_zig_exception` fills the source lines with slices of the source text. The caller owns that text and passes `&'a mut Utf8Bytes`, so the compiler checks that it outlives the holder. `ZigStackTrace::from_frames` keeps its frame buffer borrowed. - `Route`, `JSS3Error` and the npmrc test object get a lifetime. - `ResolvedSource` and `SystemError` own their strings. C++ wraps a borrowed ASCII string it receives by value with `createWithoutCopying`, so those borrows could outlive the Rust frame. - `bun bake` production moves each server chunk's bytes into the string that JavaScriptCore loads. The chunk was aliased by a string that JavaScriptCore keeps for the module's life, but `PerThread`, which owned the bytes, drops before the VM. **Cost, read from the code.** A value that ends up as a JS string is still copied once: C++ shares an owned impl and copies a borrowed one, so `clone_utf8` + share replaces borrow + copy (`process.argv`, `os.homedir()`, UDP `rinfo.address`, the `fs` `SystemError`, the inspector's bundle-failed payload). What costs more than on `main`: - The relative arm of `display_source_url_if_needed` copies the name. That is the fix. - `AsyncModule` copies `source_url`. The alias it replaces dangled once the `AsyncModule` was freed. - `from_static_bytes` (the `bun:wrap` source, a compiled module's bytecode origin path) makes a `WTF::ExternalStringImpl`: one more small allocation than `createWithoutCopying`, once per module load, freed with the string. - `PerThread::init` boxes one more key per server chunk. - Cloning a view copies its bytes. It was free, and dangling. What costs less: a builtin's resolved path is `String::static_`. In the module loader (`transferToWTFString`) a repeated import finds the atom and allocates nothing, where `main` made a `createWithoutCopying` header each time. The JS resolve APIs (`into_js`) copy it once, as on `main`. **Not changed here.** - `String::create_uninitialized_latin1` / `_utf16` return a `&'static mut` slice that lives only as long as the string. Same kind of lie, 24 callers in hot paths, so it needs its own PR. - A relative `Bun.file()` path behind a `blob:` URL stays the module's source URL: oven-sh#46842. **Also in the diff.** `test/js/web/websocket/proxy-test-utils.ts` carries the change from oven-sh#46170 unmodified. It fixes a flaky `ws-proxy.test.ts` case on Windows 2019 x64 that turned this PR's CI red and also fails on other branches. ### How did you verify your code works? - **The bug no longer compiles.** Putting `String::borrow_utf8(display)` back gives E0599. `Some(StringView::borrow_utf8(display))` gives E0308, `expected String, found StringView<'_>`. Changing the return type to `StringView<'_>` gives a lifetime error. - **Neither do these.** Declaring the source text after the holder that borrows its lines: E0597. Freeing the request body before the dev server prints its frames: E0505. Clearing the frame `Vec` while the trace points at it: E0499. - **No leak.** C++ stores an owned ref in source line 0 for an error with `lineText`. 1200 inspects of an error with a fresh 1 MB `lineText` grow RSS by 8 MB (debug build, ASAN quarantine at 1 MB). - `a cause getter that loads the throwing file again` in `test/js/bun/util/inspect-error.test.js` covers the four printers and `error.stack`. It fails with `USE_SYSTEM_BUN=1 bun test` and passes with `bun bd test` (46 pass). The repro reports the use-after-free on a debug build of `main`. - `stringViewCloneOutlivesBytes` in `test/internal/bindgen.test.ts` frees the bytes before it returns the clone. `stringViewEcho` now goes through `String::clone` for Latin-1 and UTF-16 views (226 pass). - An A/B probe of the touched behaviour gives the same output on release 1.4.4 and on this branch: `process.argv`, `os.homedir()`, brace expansion, `Transpiler.scan`, `dns.getServers()`, UDP `rinfo`, timer warnings, JSON IPC, builtin resolution (all 182 alias keys through five resolve APIs), `bun:wrap`, the valkey URL, error printing with non-ASCII source lines, system errors, redirects, S3 errors, CLI errors. - 36 related test files on a debug + ASAN build, with no ASAN report. The failures are environmental: no `USER`, no IPv6, no DNS, 5 s timeouts that pass with a longer limit, and a leak fixture that only knows an ASAN build by the name `bun-asan`. `native-plugin.test.ts` did not run locally (its setup hook timed out on a download). - `cargo check --workspace --all-targets`, `bun run rust:clippy`, `cargo fmt --check`, the source lints, and `rust:check-all` for `x86_64-pc-windows-msvc` and `aarch64-apple-darwin`. `mordant` was not run locally. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ams (oven-sh#46650) ### What does this PR do? Removes the HdrHistogram_c dependency. `createHistogram()` and `monitorEventLoopDelay()` now count in `bun_histogram`, a new crate with `#[forbid(unsafe_code)]` and no dependencies. It holds the 13 of the library's 39 functions that Bun needs. The six hand-written C++ files are replaced by three bindgen classes, so the classes stay native: every method and getter is a method of one Rust struct. **Now as in Node.js v26.3.0** - `min` and `max` are bucket edges, so `max === percentile(100)`. `add()` no longer takes a dropped value as the new `min`/`max`, and returns `undefined`. - `percentiles` holds numbers, not BigInts. - Validation and error text of `record`, `percentile*`, `add`, `createHistogram`: `record(1.5)` throws. - `Histogram`, `RecordableHistogram` and `ELDHistogram` classes, none constructible. `toJSON()` and `util.inspect()` output. - `monitorEventLoopDelay()`: a new histogram per call, without `record()`. It records the time between samples, not the lateness. `enable()` keeps earlier data. - `structuredClone()` and `postMessage()` share the counts across threads. - After `a.add(b)`, `a.recordDelta()` measures from the later `recordDelta()` of the two. It reads the clock of `uv_hrtime()`, with the counts locked, so threads that share a histogram record their times in order. - The shared `validateInteger` raised `max` to `min` when `min > max`, so `{ lowest: 10, highest: 15 }` named the wrong range. Fixed in `NodeValidator.cpp`. **Differs from Node.js, where Node or the C library has a bug** (measured on v26.3.0) | | Node.js | Here | |---|---|---| | `mean` of 2000 × `MAX_SAFE_INTEGER` | `-218371805369335.8` | `9005000231485440` | | `record(433722)`, then `percentiles.get(100)` | `433664`, below `percentile(100)` | `433919` | | `add()` that drops its only value | `count` 1, `exceeds` 0 | `count` 0, `exceeds` 1 | | `percentiles`, then `percentilesBigInt` | one `Map`, refilled | two | | Empty, `lowest: 1000`: `percentile(50)` | `511` | `0` | | `{ lowest: 1, highest: 10n }` | `TypeError` | works | | `{ lowest: 2n ** 61n, highest: 2n ** 62n }` | aborts | throws | | `add(new Proxy({}, { get: () => 1 }))` | aborts | throws | | `add()` | `TypeError` without a code | `ERR_INVALID_ARG_TYPE` | | `Reflect.ownKeys(histogram)` | `constructor` and three symbols | none | | `v8.serialize(monitorEventLoopDelay())` | does not throw, and the data is lost | `DataCloneError`, as in 1.4.4 | | `disable()`, 400 ms, `enable()` | records 420 ms | does not | Seven differences are not fixes. They come with a native class, and every native class in Bun has them: - `new Proxy(histogram, {}).count` and `Object.create(histogram).count` throw, because the counts live in the object itself. Both are printed as `RecordableHistogram {}`, where Node prints the numbers. - A getter with the wrong `this` throws a `TypeError` without a code: "The Histogram.count getter can only be used on instances of Histogram". Node throws `ERR_INVALID_THIS`. - A method with the wrong `this` throws `ERR_INVALID_THIS`, as in Node, with Bun's text: "Expected this to be instanceof RecordableHistogram, but received …". - The prototypes have `Symbol.toStringTag`, so `Object.prototype.toString.call(histogram)` is `[object RecordableHistogram]` and not `[object Object]`. - `toJSON()` reads the counts and not the public getters, which only shows if a getter is redefined on the instance. - `RecordableHistogram()` without `new` throws a `TypeError` with the code `ERR_ILLEGAL_CONSTRUCTOR`. In Node it has no code. - `console.log(Histogram.prototype)` lists the members, the getters as `undefined`. Node prints `{}`. **bindgen** `RecordableHistogram` and `ELDHistogram` inherit from `Histogram`, which bindgen could not say. It gets: - `extends`: a class whose C++ cell is a subclass of its base's and shares its Rust struct. The prototype and the constructor inherit from the base's, and an instance is an instance of the base wherever one is taken. It is what `JSS3File` writes by hand on top of the generated `JSBlob`, and how WebCore does inheritance. - `final: false` for a class whose struct is stored in the object. A subclass adds no fields, which `STATIC_ASSERT_ISO_SUBSPACE_SHARABLE` checks. - The key `@@inspectCustom`, which is `Symbol.for("nodejs.util.inspect.custom")`, and a JS builtin under an `@@` key. Two members are builtins written in JavaScript: `inspect.custom`, which calls `util.inspect`, and `[Symbol.dispose]`, which calls `this.disable()` as Node's does. A number in Node's `ERR_OUT_OF_RANGE` has its own format (`-0`, `9_007_199_254_740_992`). `bun_core::fmt::NodeNumber` prints it, for `record()`, `percentile()` and the `validate_integer` of `node:fs`, and in place of a loop that `crypto.randomInt` had. `createHistogram()` and `monitorEventLoopDelay()` validate their options in JavaScript, as before. **Memory** HdrHistogram_c allocates a slot for every value up to `highest` when the histogram is created: 360 KB by default, 38 MB with `figures: 5`. Here the counts are in chunks of 512 slots (4 KB), and a chunk is allocated when a value is first recorded in it. So the memory depends on how many different values there are, not on how large they are. | | Bytes | |---|---| | `createHistogram()` | 224 | | One value | 5 KB | | Six values from 100 to 1e9 | 25 KB | | A value in every chunk | 361 KB | | `figures: 5`, `record(1)` and `record(MAX_SAFE_INTEGER)` | 84 KB | `reset()` frees the chunks. Before, the size told to the GC was stored in a `uint16_t`. The GC is told what JavaScriptCore tells it of a `SharedArrayBuffer`, which also grows after it is created and which a structured clone also shares with other threads: | | Told to the GC | As in JavaScriptCore | |---|---|---| | An object is created, a clone too | the whole size, as an allocation | `JSArrayBuffer::finishCreation` | | An object is marked | the whole size, as held | `Heap::extraMemorySize`, `JSWebAssemblyMemory::visitChildrenImpl` | | `record`, `add`, `recordDelta` or a sample of the event loop allocates a chunk | the chunk, as an allocation in the thread that did it | `ArrayBuffer::grow`, `JSWebAssemblyMemory::growSuccessCallback` | The first two are what `estimatedSize` in bindgen already does, which is `[ReportExtraMemoryCost]` in WebCore. The third is `Heap::reportExtraMemoryAllocated`, which Rust could not call: it is now `VM::report_extra_memory_allocated`, and `JsHeap::report_extra_memory` in bindgen. So clones of one histogram each count all of it, and a thread is not told of what another thread adds, as with a `SharedArrayBuffer`. **Speed** (release builds, linux x64, the median of 3 runs) | | 1.4.4 (C) | Here | Node.js | |---|---|---|---| | `createHistogram()` | 18 µs | 0.17 µs | 14 µs | | Create, record 1e5 to 1e9, drop | 22 µs | 3.5 µs | 48 µs | | … peak RSS of 30,000 turns | 198 MB | 52 MB | 833 MB | | The same, keeping the last 2000 | 65 µs | 5.8 µs | 129 µs | | … peak RSS | 1397 MB | 165 MB | 3045 MB | | `mean`, 2 values far apart | 179 µs | 0.47 µs | 184 µs | | `mean`, dense | 161 µs | 35 µs | 171 µs | | `stddev`, dense | 468 µs | 77 µs | 467 µs | | `percentile(99.9)`, dense | 8.1 µs | 3.8 µs | 8.2 µs | | `percentiles`, dense | 124 µs | 27 µs | 129 µs | | `add`, dense | 166 µs | 66 µs | 173 µs | | `structuredClone()`, every chunk in use (360 KB) | throws | 25 µs | 6.6 µs | | … with `figures: 5` (37 MB) | throws | 144 µs | 6.6 µs | These were measured while the classes were written in JavaScript. `record` (13 ns, 7.3 ns in 1.4.4) and `count` (7.4 ns, 3.2 ns) each went through a JS method then, which they no longer do, and have not been measured since. The rest is time spent in Rust. With `figures: 5` the first `record` also allocates a 76 KB table of chunks. Slower than Node.js: a clone of a large histogram, because it counts as an allocation of the whole size and so makes the GC run. That is the cost of a clone of any shared buffer here: a 37 MB `SharedArrayBuffer` takes 136 µs in the same build. `packages/h3blast` linked the library's objects, so it gets a small histogram of its own. ### How did you verify your code works? - `test/js/bun/perf_hooks/histogram.test.ts` uses `node:test`: 98 pass here, 48 fail on Bun 1.4.4. Under Node v26.3.0, 85 pass and the 13 for the table above are skipped, so its expected values are Node's. It passed unchanged when the classes went native, but for the texts above. - `test/internal/bindgen.test.ts` covers `extends` with a class of its own: the two chains, a base member on an instance of the subclass, a member of the subclass on an instance of the base, `from_js` both ways, the internal fields of the base, `Drop`, and what a schema may not say. - Node's `test-perf-hooks-histogram.js` is added unmodified. It passes, and fails on 1.4.4. - `test/js/node/perf_hooks/perf_hooks.test.ts` checks the sizes above, and that histograms are collected without an explicit GC once they are dropped: those that grew, clones, and those that arrived from a worker. - The fuzzer below is not in the repo, so I planted 19 bugs in the chunks and the binding (a chunk skipped, a slot index off by one, `reset()` that keeps counts, a wrong size, the GC not told of growth, the disabled time recorded, deltas rounded to microseconds, the clock read before the lock, …) and ran only these two files: each bug fails at least one test. That was before the classes went native and has not been repeated since. - Real packages, under an idle and then a blocked event loop, on this branch, Node v26.3.0 and Bun 1.4.4: `prom-client`, `@opentelemetry/instrumentation-runtime-node`, `@fastify/under-pressure` and `piscina`. This branch reports what Node reports. 1.4.4 differs: an idle loop at `resolution: 10` has a p50 of 0.1 ms there, and 10.2 ms here and in Node. - A differential fuzzer outside the repo compares the crate with HdrHistogram_c 0.11.10, the same under UBSan, 0.12.1, Node's 0.11.9 and a naive model, and the JS API with Node v26.3.0. - Over the versions of the crate: about 1.3 million random programs, 2,064,400 configurations, 586 million slots, every reachable branch covered, and mutation testing. More than 110,000 random API programs against Node with no difference outside the table above. Also threads, GC stress, forged serialized bytes, `node:vm` realms, tampered builtins, the top JIT tier, and 54 kinds of object printed in 60 ways. - All of this was before the classes went native. The crate has not changed since. - The campaigns that go through the JS API were repeated on the native classes, at abf5d8c: no crash, no sanitizer report, no wrong number. - 3,000 random API programs against Node, with no difference outside the lists above. 60 rounds of up to 61,000 values with whole `percentiles` Maps and 400 `percentile` calls each are identical to Node. - 2,464 calls with hostile arguments and receivers, most of them throwing natively: no unchecked exception under `BUN_JSC_validateExceptionChecks=1`. - Every member called 200,000 times and then with a call that breaks the speculation, and 20 hot readers and writers beside a worker that fills and resets shared counts: also with `jitPolicyScale=0`, `useZombieMode`, `validateDoesGC`, `validateGraph` and `collectContinuously`. - The GC, clone and thread campaigns give the memory figures they gave before, equal to their `SharedArrayBuffer` controls. 110 exits racing workers under LeakSanitizer: none bad. - It found two bugs, both fixed in later commits and both now tested: printing an object that inherits from or proxies a histogram threw, and `record(-0)` said "Received 0". - The commits after abf5d8c (those two fixes, `NodeNumber`, and fixes from a review) are being fuzzed now. - It found six bugs of mine, all fixed here: `percentile(100)` below `max` at huge counts, four in how the counts were allocated, which is what led to the chunks, and a histogram that reached a thread empty and then full, of which its GC was told nothing (over 5 GB where it is now 0.7 GB; Node.js passes 5 GB too). That one came from a rule of my own for clones, which is gone. - All of it was repeated on the chunks, the last change to the crate, and found nothing: 370,000 random programs, the same sweeps, 3.85 million ranks, every reachable branch, 490,000 coverage-guided runs and 74,000 API programs. Each check also asserts that a chunk exists if and only if one of its slots is in use, and that the size told to the GC is exact. Of 317 mutants of the crate, 20 do not compile, 294 are killed, and the 3 that survive cannot change a result. - The histogram of `h3blast` gives what HdrHistogram_c gives for every value from 0 to 60,000,010 and in 3000 random rounds, bit for bit, under ASAN and UBSan. - macOS and Windows are untested locally. One thing is specific to macOS: `CLOCK_MONOTONIC` has whole microseconds there, which is why `recordDelta()` reads `uv_hrtime()`. The test for it fails on Linux when the deltas are rounded to microseconds, but only the macOS lanes can show that the real clock is fine. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
### What does this PR do? Moves `WEBKIT_VERSION` from `680ef47cec2c` to `982277c9dd3e`, the head of oven-sh/WebKit `main`, and updates the places in Bun that the new engine needs changed. The range is oven-sh/WebKit#757, oven-sh#903, oven-sh#905, oven-sh#906 and oven-sh#907. This PR does not add or enable JavaScriptCore's ahead-of-time compiler. That is oven-sh#45631. **Build scripts** - **Prebuilt names:** oven-sh/WebKit#757 renamed the release assets. `release`, `lto` and `lto-hardened` keep their names and are built with `ENABLE_AOT=0`. Every other variant now ends in `-aot`, so `prebuiltSuffix()` appends that. In CI this changes one lane, `linux-x64-asan`. Locally it changes `bun bd`. - **Debug warnings:** `AssemblyHelpers.h` captures `this` in generic lambdas, which `-Werror=unused` rejects in debug builds, so `-Wno-unused-lambda-capture` is added there. **Code** - **Removable features:** JavaScriptCore now calls `Bun__removableFeatureState` and `Bun__throwForRemoved*`, in every variant. `RemovedFeature.cpp` defines them the way WebKit's own `AOTNoFeatureRemoved.cpp` does: nothing is ever removed. - **Table of built-ins:** the `-aot` libraries leave 35 symbols (`JSC::AOT::builtinTable*` and `describedBuiltinTable`) for the program that links them to define. Only the compiler and the code it compiles read that table, and `Options::useAOT()` is off, so `RemovedFeature.cpp` defines an empty one under `#if ENABLE(AOT)`. - **Built-in functions:** `BuiltinGenerator` returns `JSC::BuiltinCode`, and `BuiltinsSourceProvider::create` takes the id of its first function. - **`JSType`:** five new types moved the numbers mirrored in `JSType.rs`. A `static_assert` now pins them. - **Static strings:** `StringImpl::ref()` and `deref()` no longer count static strings, which may be in read-only memory. The Rust port of the two does the same. Three `ASSERT(refCount() > 0)` in `BunString.cpp` accept a static string, whose count is now 0. `RemovedFeature.cpp` has the path oven-sh#45631 uses, so that PR gets a merge conflict instead of duplicate symbols at link time. `JSType.rs`, `bun_alloc/lib.rs` and `BunString.cpp` are the same hunks as in oven-sh#45631. ### How did you verify your code works? Bun was not built locally; CI is the build. - **CI build [#133439](https://buildkite.com/bun/bun/builds/133439)**, on an earlier commit of this PR with the same engine changes: all 13 `build-bun` jobs passed, ASAN included. That commit generated a full table of built-ins. The empty table that replaced it has not linked in CI yet. - **New test:** the three cases of `webkit-upgrade-982277c9dd.test.ts` fail on main's CI binary at the merge base (old engine) and pass on the linux-x64 binary of #133439 (new engine). - **Bytecode snapshot:** `BytecodeList.rb` changed, so six values in `bundler_bytecode_portable.test.ts` moved. They come from the linux-x64 binary of #133439, and linux-aarch64-musl reported the same six. Checks done without building: - **Embedder symbols:** `llvm-nm` over the old and new `linux-amd64` archives and the new `linux-amd64-asan-aot` archives gave the 40 symbols Bun has to define. `RemovedFeature.cpp`, compiled alone with Bun's flags, defines all 40. - **The table is not read:** in the engine outside `aot/`, the AOT hooks in VM, heap and realm setup that I read are behind `vm.hasCompiledCode()` or `Options::runsCompiledImage()`, and only `$vm` names the table. - **Assets:** all 30 asset names Bun can ask for exist at the new tag. - **Headers:** I diffed the 225 changed headers and searched `src/` for every removed or changed name. - **`JSType.rs`:** a script compared all 88 constants with `JSType.h` at the pinned commit. - **Syntax check:** `clang++ -fsyntax-only` with the flags from `compile_commands.json` passes on `RemovedFeature.cpp` (with and without `ENABLE_AOT`), the generated `WebCoreJSBuiltins.cpp` and 29 hand-written files that use the changed APIs. - **arm64 allowlist:** the outline-atomics helpers the arm64 archives use are the same set as before, so `allowlist-aarch64.txt` does not change. - **Lint:** `test/internal/source-lints` passes (184 tests), `tsc` passes for `scripts/build` and `src`, and Prettier, clang-format and `cargo fmt` are clean. Not covered: - `--webkit=local`, Windows and macOS debug builds, and `--hardened=on` are untested, and CI builds none of them. **Binary size:** the binaries grow by 0.75 to 1.64 MB (linux-x64 +1.39 MB, darwin-aarch64 +1.17 MB, windows-x64 +1.08 MB), over the 0.5 MB limit. It is JavaScriptCore's own growth: release builds already link the prebuilt without the ahead-of-time compiler, and Bun's diff adds a few hundred bytes. The head commit carries `[skip size check]`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…#46638) ### What does this PR do? Removes the dependency on libarchive. The two things Bun used it for, tar and the gzip container, are now written in safe Rust. - **`src/tar` (`bun_tar`, new, `#[forbid(unsafe_code)]`)** - `Reader`: a push parser. The caller hands it whatever bytes it has and gets events back (`Entry`, `Data`, `EndOfEntry`, `EndOfArchive`, `NeedInput`), so running out of input is an ordinary state instead of the `ARCHIVE_RETRY` patches. Reads v7, ustar, pax, GNU long name/link and every GNU/SUN sparse format, in the two dialects Bun had: the npm one (node-tar rules) and the general one (concatenated archives). - `Writer`: restricted pax, regular files only. - `gzip`: header, trailer, multi-member and nested streams. DEFLATE itself still comes from zlib through `bun_zlib`. - `compose`: what Windows does to a UTF-8 pax path (composition only, not NFC, Unicode 6.0 tables). - What differs by platform (charset, AppleDouble `._` files) is a runtime option, so all of it compiles and can be exercised on any host. - **`src/libarchive` becomes `src/archive` (`bun_archive`)** and keeps only the extraction policy: path safety, symlinks, modes, sparse writes. - `Bun.Archive`, `bun install` (buffered and streaming), `bun pm pack`, `bun publish`, `bun pm diff` and `bun create` are ported. `TarballStream` loses its C read callback and phase machine, and its drain-only state moved into its own struct so no reference to the whole struct is held while the HTTP thread appends. - Deleted: the dependency, its six patches, the update workflow and `process.versions.libarchive`. The attribution stays in `LICENSE.md`, because parts of `src/tar` are derived from libarchive's tar reader and pax writer. The goal was no observable change, quirks included. These are the changes I know of: | Change | Why | | --- | --- | | ACL records (`SCHILY.acl.*`, Solaris `A` headers) no longer change an entry's permissions | Bug fix. An archive made with `tar --acls` could lose its exec bits or extract a directory as mode 000. GNU tar and node-tar take permissions from the header | | A malformed ACL record is no longer a warning | Follows from the row above. libarchive trusted an entry less after a warning: a regular entry whose name ends in `/` stayed a file, with its data. Now it becomes a directory, as it does after any well-formed header, and what was its data is read as headers. Only seen in generated archives | | An in-memory entry of more than 2 GiB − 1 bytes is written whole, by `new Bun.Archive({...})` and for the `package.json` that `bun pm pack` and `bun publish` print again | Bug fix. libarchive takes at most `INT_MAX` bytes per write call, and Bun made one call and did not look at the count. The header gave the full size, the rest of the entry was zeros, and there was no error. Measured with release binaries: for an entry of 2 GiB + 1,024 bytes, the last 1,025 bytes are zeros on main and right here. No test: it needs about 4 GiB of memory | | A pax record whose `<length> <key>=` is longer than 512 bytes, in a pax header of more than 1 MiB, inside gzip, is always read | libarchive looked for the `=` only in what was left of its 64 KiB block of unpacked bytes (or in 512 bytes if that was more), and otherwise passed over the rest of the header. So the same tar gave one name compressed and another uncompressed or shifted by a byte. Needs a key of about 506 bytes or more. The same on Windows, in nested gzip and in several members: only the position in the innermost unpacked stream counts. Found by measuring libarchive's own coverage | | A tarball extracted while it downloads no longer fails because of where a piece of the download ends | Bug fix. The patch that let libarchive wait for more input did not cover every read: the optional fields of a gzip header, an extension body over 1 MiB, and, outside npm's rules, more blocks of an old GNU sparse map, a sparse 1.0 map and a `._` file's contents. A piece that ended inside one of those failed the install with "Truncated tar archive". Reproduced with real `bun install` on main for tarball URLs and `github:` dependencies: six of six cases fail there and pass here, and the same tarballs install on main when they are not streamed. Only tarballs of 2 MiB and more are streamed by default, so it takes an unusual one. Every cut position of six small tarballs through real installs, 22,704 per binary: main fails at 1,064, exactly the ones a model of it in the harness predicts, and this branch at none. In that harness, 7.96 million pairs of libarchive with and without a read callback that runs dry: 61,080 fail only when it does, none fails only when it does not, and none succeeds both ways with different entries | | `Bun.inspect(archive)` counts files past a block with a bad checksum | It now agrees with `files()` | | Some error messages for damaged archives | Measured against libarchive on generated damaged archives: about 96% are word for word the same for plain tar. Most of the rest are cases where libarchive set no message (surfaced as `ReadError`) or left a stale warning in place. In `Truncated input file (needed N bytes, only M available)`, `M` is now the real count; libarchive printed 0 in nearly every case | | The verbose label `libarchive error:` is now `archive error:` | | | `process.versions.libarchive` is gone | | | Windows only: a name that cannot be composed (more than ten combining marks in a row) is written as raw bytes with `hdrcharset=BINARY` | `new Bun.Archive({...})` used to throw `ArchiveHeaderError` for it. Measured with CI's Windows binaries under Wine | ### How did you verify your code works? **New tests through public APIs.** The expected results were recorded with the last build that used libarchive (`record.ts` next to each fixture re-records them), so they compare Bun before with Bun after. They pass on a released 1.4.4 by design, except the ACL and `Bun.inspect` tests, which cover the intended changes. - `test/js/bun/archive-golden`: about 1,250 archives, well formed and damaged, plain and gzipped, through `Bun.Archive` `files()` and `extract()`; 21 writer snapshots through `bytes()`; entry names and written headers for Windows and macOS, with values measured on released Bun on those systems; gzip inside gzip (24 layers and no more, headers that span blocks, the 1 MiB name limit, how far an outer layer is read, a member that ends exactly on a 64 KiB block); header checksums over signed and unsigned bytes. - `test/cli/install/tarball-golden`: 191 tarballs through real `bun install`, extracted while they download and from memory. The server holds back the rest of each body until the extractor has started, so streaming does not depend on timing. - `bun-pack.test.ts`: the decompressed bytes and gzip header of a `bun pm pack` tarball. - `bun-install-streaming-extract.test.ts`: a piece of the download that ends in an optional field of the gzip header, or in a pax header of more than 1 MiB. Both fail on main. - `archive.test.ts`: the ACL fix, the `Bun.inspect` count, and the `File`s that `files()` returns: read in every way a `File` can be, printed, sliced, written to disk, and used after the archive is collected. The archives were picked from a structure-aware generator, keeping one only if it made the old reader report something no earlier one had. **New tests of the reader and writer themselves** (`test/js/bun/archive-golden/bun-tar.test.ts`). `Bun.Archive` shows only regular files, read by the rules of the system it runs on, and writes them with one mode and the current time. So `bun:internal-for-testing` gets two functions, which nothing else can reach: - `readTar(bytes, options)` returns everything the reader reports as text: type, mode, size, time, name and link target of every entry, where its data lies, and how the archive ends. The options are npm's rules or the general ones, the behaviour of macOS or Windows, reading from memory or like a download, the size of the pieces the input comes in, and which entries are skipped. - `writeTar(files, names, contents, options)` takes names as bytes, any mode, time and announced size, the naming of POSIX or Windows, and a gzip level. No Bun that used libarchive has them, so the expected results could not be recorded with one. Instead each was compared with libarchive itself, linked at Bun's pin and patches, when it was recorded: - 2,482 archives in 48 modes, 119,136 logs, all equal to what libarchive reports, wording of errors aside, and but for the permissions in 240 logs of seven archives with ACLs (first row of the table above). 1,463 archives are stored. 1,019 are made by the test: gzip cut, changed byte by byte, with every combination of header fields, in several members, nested up to 26 times, and ending on and around a 64 KiB block; values at and one over each limit of the reader; every sparse format; names that Windows composes or cannot; 19 shapes that a fuzzing corpus reached and the other archives did not; 77 written for lines and branches of libarchive's own tar reader that nothing else made it run. - The same archives with the input cut into pieces of 1, 7, 511, 512 and 513 bytes give the same logs. - 656 sets of files, each byte for byte what libarchive writes: 319 with the names of Windows, 231 compressed at levels 0 to 9, 35 with sizes around 8 GiB. - 18 long names for what Windows composes: all 931 pairs of the table among 70,000 pairs that do not compose, every character of the first two planes between a base and a mark, every mark of Unicode 6.0 against one mark of each combining class in both orders, each character of the blocks with marks where a class it should not have would show, and all Hangul syllables. libarchive read the same 18 names, and the writer has to record what the reader makes of each. A changed pair, block or class in the tables each failed a test when tried, also a class moved to a neighbouring one (27 to 29, 220 to 222, 230 to 232). - A dozen short tests that state the rules in the open, with answers libarchive also gave. Measured by replaying them through an instrumented `bun_tar`: the archives reach at least 96% of the lines of `reader.rs` and all of `header.rs`, and the file sets 96% of `writer.rs`. They run on every platform in CI, since the platform is an option. Before those names were added, the file took 1.4 s on CI's Windows machines, 5.7 s on macOS and 6.0 s under ASAN. Coverage of our own code cannot show a quirk of libarchive that no input ever sets off, so libarchive's tar reader (`archive_read_support_format_tar.c` at Bun's pin and patches) was measured too, in all 48 modes and six chunk sizes. The archives in this PR made it run 84.4% of its lines and 76.9% of its branches before the 77 were added, and 91.1% and 87.7% with them. Everything used while developing (80,102 files, the generators, a read callback that runs dry, and inputs written for this) reaches 92.3% and 88.6%, and all 55 functions. Each of the 198 lines left was read: an option Bun never set or a state it never creates (17 groups), another platform (6), a failed allocation (21), or dead code (23). This is a Linux build, so code for Windows and macOS alone is not compiled and not measured. 14 groups of lines and 4 branches could be reached and never had been. 1,108 inputs now reach all of them, and 0 of 52,944 logs differ. The pax row in the table above came from a branch that line coverage had counted as covered. **While developing**, with libarchive at Bun's pin and patches linked into the same process as `bun_tar` (not in this PR): - Reader: over 30 million comparisons of the full event log in twelve modes (both dialects, detection on and off, macOS and Windows behaviour), each also checked for not depending on how the input is cut into pieces. That includes skipping entries, gzip members and trailers of every shape, and up to 26 nested gzip layers, compared for exact equality, down to which entries are seen before a failure. - Writer: 33,000 comparisons, plain and at gzip levels 0 to 9, byte for byte. - Name composition: every code point pair that can compose, plus 9 million random strings. - A coverage-guided differential fuzzer (libFuzzer with a structure-aware mutator, every execution runs every mode): over 4 million executions, seeded with generated archives, archives made by other tar tools, libarchive's own test fixtures and npm tarballs. The harness reaches 96% of the lines of `bun_tar`; what is left is allocation failure and the unreachable side of checked indexing. **Old binary against new binary on Linux:** - About 11,300 archives through `Bun.Archive`: `files()` identical, `extract()` different only for ACL permissions. That includes what a rejected extraction leaves on disk. - About 9,000 generated tarballs through `bun install` as `file:`, buffered and streaming: same trees, same refusals. - `test/package.json` from its lockfile, 1,191 real packages served by a local registry: the two `node_modules` trees are identical (67,251 paths with mode, size and content hash). - 6,300 archives from a generator of hostile paths (symlink chains, `..`, absolute paths, directories swapped for files), half as root and half as an unprivileged user: no difference and nothing written outside the destination. - `bun pm pack` at every gzip level, and 4,000 random `new Bun.Archive({...})` inputs: identical bytes. - Five kinds of failing archive, 9,000 operations each in one process: no file descriptor left open, same memory growth. **Old binary against new binary on Windows (CI's x64 binaries under Wine):** 600,000 reads and 12,000 extractions with no difference; 150,000 writes that differ only in the row above. The existing archive, streaming-extract, tarball-integrity, symlink-traversal, pack, publish, pm-diff, cache-pack and create suites pass locally on Linux with main merged in, except `bun-create.test.ts` `should not crash > ["create"]`: in my container a bare `bun create` exits 0 with the build from before this PR too. CI has since run all of it on Windows, macOS, musl and ASAN. ### Performance CI's linux-x64 release binaries, main at the merge base against this branch, median of 11 runs on a memory-backed file system. `small` is 20,000 files in 100 MB (pax headers), `big` is one 200 MB file. | | main | this PR | | | --- | --- | --- | --- | | `files()` small.tar | 95.8 ms | 91.4 ms | 0.95 | | `files()` small.tar.gz | 61.2 ms | 52.7 ms | 0.86 | | `files()` big.tar | 469 ms | 316 ms | 0.67 | | `files()` big.tar.gz | 493 ms | 488 ms | 0.99 | | `extract()` small.tar | 188 ms | 175 ms | 0.93 | | `extract()` small.tar.gz | 137 ms | 134 ms | 0.98 | | `extract()` big.tar | 252 ms | 243 ms | 0.96 | | `extract()` big.tar.gz | 277 ms | 281 ms | 1.02 | | `new Bun.Archive({20,000 files}).bytes()` | 140 ms | 120 ms | 0.85 | | the same with `compress: "gzip"` | 210 ms | 170 ms | 0.81 | | `bun pm pack`, 20,000 files | 696 ms | 679 ms | 0.98 | | `bun install`, 1,191 packages, cold cache, local registry | 1006 ms | 1017 ms | 1.01 | Most of that time is zlib, system calls and JavaScript objects, which did not change. The readers alone, in one process at the same optimization level: walking an archive of plain headers takes 0.34 times as long as with libarchive, pax headers 0.53, a `.tgz` 0.61. Archives built to be expensive (millions of pax records or sparse regions, 256 MB of junk, a 1 GiB bomb under 24 gzip layers, a million empty gzip members) all take the same time as before or less. Anonymous memory is the same. The release binary is 64 KB to 130 KB smaller, depending on the target. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do?
Removes `bytemuck` from the Rust dependencies. A small in-tree crate,
`bun_pod` (`src/pod/`, `#[no_std]`), replaces it, and `bun_core`
re-exports it, so call sites read `bun_core::cast_slice`,
`bun_core::bytes_of` and so on. The trait names are the same:
`NoUninit`, `AnyBitPattern`, `Pod`, `Zeroable`. `src/pod/README.md` is
the crate's documentation and holds its doctests.
#### What is better than bytemuck
- **Trait impls are proven by the compiler.** `impl_no_uninit!`,
`impl_any_bit_pattern!`, `impl_pod!` and `impl_zeroable!` are safe
macros. Each names a function whose only content is a
`core::mem::TransmuteFrom` bound, so rustc proves the claim from the
type's real layout on the target being compiled. Padding, a `bool`, an
enum or a pointer in the wrong place is a compile error at the macro
call. Every `unsafe impl bytemuck::*` is gone, and most `unsafe impl
Zeroable` for libc and Win32 structs are proven too.
- **An empty slice always casts.** bytemuck checked the alignment of the
dangling pointer of an empty `&[u8]` and panicked.
- **Panics say more.** They carry the caller's location
(`#[track_caller]`) and the address or length at fault. The panic
function is not generic, so there is one copy.
- **One `Zeroable` instead of three** (bytemuck's, `bun_core::ffi`'s,
and a private copy in the Windows shim).
- **Casts to bytes are `const fn`.**
- **Release code is the same.** In release assembly `slice_as_bytes`,
`bytes_of` and `cast_slice::<u16, u8>` fold into the same function as
bytemuck's versions, and the hot path of `cast_slice::<u8, u16>` is the
same eight instructions.
#### A crash this fixes
An `ArrayBuffer` can sit at any address (`bun:ffi` `toArrayBuffer`,
N-API external buffers). Printing a typed array wider than a byte over
one at an odd address panicked:
```js
import { ptr, toArrayBuffer } from "bun:ffi";
const bytes = new Uint8Array(64);
bytes.buffer;
console.log(new Uint16Array(toArrayBuffer(ptr(bytes) + 1, 0, 8)));
// 1.4.4: panic: cast_slice>TargetAlignmentGreaterAndInputNotAligned
```
`console.log`, `Bun.inspect`, the `expect()` diff printer and
`Bun.unsafe.arrayBufferToString` now read such memory through
`Unaligned<T>` (`Unaligned::slice_of`, `String::clone_utf16_bytes`).
`slice_of` has no alignment to check and leaves out a trailing partial
element, so it cannot fail. `ArrayBuffer::as_u32`, which `node:zlib`
uses for its write state and params, returns `&mut [Unaligned<u32>]`;
before, it formed a `&mut [u32]` behind a debug assertion only.
#### Shapes that no longer need a cast
- Sinks take UTF-16 text as `write_utf16(&[u16])`. One caller turned
`&[u16]` into bytes, and seven sinks cast it back with a check that
could panic. The fetch and S3 sinks share one `EncodedChunk` enum
(bytes, Latin-1 or UTF-16), now in `streams.rs`.
- The lexer keeps the raw string literal in one enum
(`StringLiteralRaw`) instead of bytes plus a format tag.
- Code generic over `u8`/`u16` dispatches through `PathByte` methods
(`units()`, `index_of_unit()`, `last_index_of_unit()`) instead of
`cast_slice::<T, u8>` behind an `IS_U16` check. This also replaces
`strings::Lanes` and `paths::string_paths::Ch`.
- `write_any_to_hasher` takes any `NoUninit` value. `AsBytes` and two
local copies of the function are gone.
- `BinLinkingShim::encode_into` writes into `&mut [u16]`, which removes
two `unsafe` blocks and the aligned wrapper struct in its caller.
- The zlib, brotli and zstd contexts return `write_result() ->
[avail_out, avail_in]`. The out-parameters existed to write straight
into the JS `Uint32Array`, which is no longer a `&mut [u32]`.
- `parse_hex_to_int::<T>` was only used with `u64` and is now
`parse_hex_to_u64`. `SignalCode::from::<T>` had one caller, which passed
a `u8`, and is gone.
#### Functions that were safe in name only
These took `T: Copy` (or nothing) and reinterpreted bytes. They now
carry the bound that makes them safe:
- lockfile `read_array<T: AnyBitPattern>` / `write_array<T: NoUninit>`.
The no-op `assert_no_uninitialized_padding` is deleted, and `read_array`
no longer depends on the alignment of the buffer.
- the manifest cache's `read_array` / `write_array`. `NpmPackage` is now
`Pod`, which replaces its hand-written `offset_of!` asserts. For that,
`has_extended_manifest` is a `u8` with a `has_extended_manifest()`
accessor. The on-disk layout is unchanged.
- `exe_format::read_struct` / `write_struct`,
`FixedBufferStream::read_struct`, Mach-O `LoadCommand::cast`.
- The Mach-O `LoadCommandIterator<'a>` borrows its buffer and implements
`Iterator`. It used to hold a raw pointer (`RawSlice`) so that two loops
in `macho.rs` could write to the same `Vec` while walking it, one of
them through a pointer derived from a shared borrow. Those loops now
walk owned ranges (`command_ranges()`) and edit through slices. The
walker has no `unsafe` left, and the compiler rejects the old pattern.
#### What to know before merging
- **One more nightly feature:** `#[feature(transmutability)]`, enabled
only in `bun_pod` and used by three functions in its `proof` module.
Crates that call the macros do not need it.
- **The proof covers soundness, not field order.** rustc accepts a
`repr(Rust)` struct when the field order it picked leaves no padding,
and the bytes then follow that order (`{ a: u8, b: u32, c: u8, d: u16 }`
comes out as b, d, a, c). A `macro_rules!` macro cannot see a `repr`, so
a new source lint, `test/internal/source-lints/pod-impl-repr.test.ts`,
requires a `#[repr(..)]` on every type named in the three byte-view
macros. All current types have one.
- **The proof does not know library invariants.**
`impl_any_bit_pattern!` accepts a wrapper around a type whose private
fields carry an invariant, even a type from another crate. bytemuck's
derive closes this with a bound on each field, which needs a proc macro.
No type in the tree is affected, and the README says not to do it ("What
the proof leaves to you").
- **What the proof cannot analyze is a compile error, never a wrong
answer.** It cannot analyze pointers, generic parameters, or a field
whose type is an associated type (this includes `bitflags!` storage).
- **Types over 4 KiB are refused.** The analysis recurses once per byte,
and rustc segfaults somewhere between 16 and 32 KiB. The macros fail
with a message before the analysis starts.
- **`bun_pod` has one feature, `alloc`,** for `vec_from_bytes`.
`bun_core` turns it on. The freestanding Windows shim, which is built
with `core` alone, does not.
- **Some `unsafe` stays, with a comment saying why.** `VersionType`,
`ExternalSlice<T>`, `EventListBytes`, `Dict` and the structs with
pointer fields keep an `unsafe impl`. `Offsets` and `PackageVersion`
keep their `unsafe` byte views (`PackageVersion` holds `bin::Value`, a
union whose arms differ in size).
### How did you verify your code works?
- **New tests.** The crashing inputs run in a spawned child, so an
unfixed build fails one named case each.
- `test/js/bun/util/inspect.test.js` "TypedArray over an ArrayBuffer at
an odd address prints" (all nine typed arrays wider than a byte) and
`test/js/bun/util/unsafe.test.js` "arrayBufferToString u16 at an odd
address" fail on Bun 1.4.4 with the panic above and pass on this branch.
- Two tests pass on 1.4.4 as well and guard the new code instead.
`test/js/node/zlib/zlib.test.js` "native streams take a write state and
params at an odd address" covers the `Unaligned<u32>` path in zlib,
brotli and zstd. A release build of 1.4.4 skipped the debug assertion
there and did unaligned stores. `unsafe.test.js` "arrayBufferToString
u16 of nothing" covers the empty-slice cast.
- **`bun_pod`.** 12 unit tests and 12 doctests, 7 of them `compile_fail`
(padding, an uneven union, a pointer, an enum, a `bool`, a non-zero
enum, an oversized type). The crate is added to `MIRI_CRATES`, so CI
runs all of them under Miri, and `rust-lints.yml` now also runs when
`src/pod/README.md` changes.
- **Mach-O output.** `bun build --compile --target=bun-darwin-arm64`
with the 1.4.4 darwin binary as the template and payloads of 20 B, 300
KB and 5 MB gives a signed file byte-identical to what Bun 1.4.4 writes.
Five malformed templates (zero, tiny and huge `cmdsize`, too many
`ncmds`, a short `LC_SYMTAB`) give the same result on both: four fail
with the same error and one gives identical output.
`compile-macho-codesign.test.ts` passes.
- **Type checks.** `cargo check --workspace` for Linux (through clippy,
`--all-targets`), Windows x64 and macOS arm64 on this commit, plus the
standalone Windows shim without `alloc`. FreeBSD, Android and musl
passed on an earlier state of the branch, and CI built all 12 targets on
the first commit.
- **The new lint.** It has a self-test with four synthetic sources, and
removing `#[repr(C, packed)]` from `SettingsPayloadUnit` makes it fail
with that type's name.
- **Lints.** `bun run rust:clippy`, `cargo fmt --check`, prettier and
`test/internal/source-lints/` pass.
- **Test suites on the debug build.**
- On this commit: `inspect`, `unsafe`, zlib (three files), `node:path`,
`test/js/bun/sourcemap`, `node:module` sourcemap, and the dev server's
sourcemap test.
- On the previous commit: `bun-lockb`, `migrate-bun-lockb-v2`,
`arraybuffersink`, `filesink`, `body-stream`, `fetch-abort-stream-body`,
`s3-networksink-leak`, `s3-connection-close`, `compile-corrupted-embed`,
`bun-install-cache-publish`.
- On earlier builds of the change: `blob`, `streams`, `s3`, `serve`,
`node-http2`, `buffer`, `transpiler`, `transpiler-cache`, `yaml`, `xml`,
`bundler_compile`, `bun-install-patch`, `html-rewriter`.
- **Failures seen locally.**
- These also fail on Bun 1.4.4 in the same container: the HTMLRewriter
"upstream body fails" group (no IPv6 there), some `node-http2` tests,
two S3 credential tests and the root-port test in `serve.test.ts`.
- Two tests ran past a 5 s limit on the debug build, and I did not
compare either with a baseline. `xml.test.ts` "deep values are a
catchable error" took 7 s. The child test in
`pretty-format-overflow.test.ts` took 5.3 s. Neither reaches code this
PR changes, and both files passed in CI on the first commit.
- `bun-build-compile.test.ts` did not finish within 280 s on the debug
build.
- **Compared with Node and Bun 1.4.4.** `node:path` `join`, `normalize`,
`resolve`, `relative` and `format`, posix and win32, on inputs from 1 to
70,000 segments (every pool tier and the heap spill) give the same 30
lines on all three. Streaming 200 KB through deflate, brotli and zstd
with a 64-byte `chunkSize` gives the same output and byte counts as
1.4.4. An already-transpiled UTF-16 source with two `sourceMappingURL`
comments remaps through the last one, as on 1.4.4.
- **By hand on the debug build.** `bun install` treats a manifest cache
file cut to 0, 1, 40, 60, 100, 150 or all but one of its 664 bytes as a
cache miss and installs. ASCII, Latin-1, UTF-16 and byte chunks through
a streaming `fetch` body and an S3 `writer()` arrive as the right UTF-8.
A failing `toEqual` prints an odd-address `Uint16Array` in its diff.
UTF-16 text through `ArrayBufferSink` and a direct `ReadableStream`, a
UTF-16 BOM `Blob` at even and odd offsets, `ucs2` writes to an odd
offset, and JSX entity strings all give the expected output.
- **Not run locally.** The crash handler's use of the load-command
iterator (macOS only) and the Windows-only code paths (the shim encoder,
`resolve_windows_t`'s drive-cwd lookup, the `PathBuffer` wide views)
were type-checked only. On the first two commits, CI passed every
Windows test job and one of the two macOS arm64 test jobs. The other
macOS arm64 job timed out both times downloading its build artifact,
before any test ran, and the two macOS x64 jobs never got an agent.
---
_Generated by [Claude Code](https://claude.ai/code)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do? Moves `WEBKIT_VERSION` from `982277c9dd3e` to `191c6a41380b`, the head of oven-sh/WebKit `main`. > [!NOTE] > The release `autobuild-191c6a41380bb1e2e54be4b6e50ecda83b189f96` was published at 10:39 UTC on 2026-10-27, after this went up. Builds #133617 and #133628 ran before that and failed where they download WebKit (HTTP 404). They say nothing about the change. The PR needs one new build. What the range brings: | Fork PR | What it changes | | --- | --- | | oven-sh/WebKit#912 | The ahead-of-time compiler (890 of the 942 files). Bun does not use it. Outside `aot/`: B3, the marked block cage and its reserve, `Interpreter::getStackTrace`, `ProxyObject`, `JSObject::putDirect`. | | oven-sh/WebKit#916 | The hardened Linux arm64 lanes are compiled without `-mtune`, so that glibc's `_FORTIFY_SOURCE` wrappers are inlined. | | oven-sh/WebKit#917 | The buffer of an old `WeakMap` or `WeakSet` that has shrunk counts until the next full collection. | The pin is the only change Bun needs: - `Heap::didChangeVisitedExtraMemory` is gone. Nothing in Bun calls it. - `Interpreter::getStackTrace` has one more parameter, at the end and with a default. Bun's five callers compile as they are. - The empty table of built-ins in `RemovedFeature.cpp`, which the debug and ASAN builds link: the range adds or removes no `builtinTable*` symbol, no entry of the two list macros and no `extern "C"` hook for the embedder. `BuiltinTable::typedArrayTypeOf`, which is new, is defined in the engine. - No asset is renamed in `lanes.mjs`, and no toolchain pin moves. - `BytecodeList.rb` is unchanged. The one change in the bytecompiler is for private names in object literals, which only built-in code can write, so the hashes in `bundler_bytecode_portable.test.ts` should stay. ### How did you verify your code works? Nothing was built. The list above is from reading the diff of the two commits against Bun's sources. The release has the same 77 asset names as the one of the old pin, all uploaded, and among them the 13 that the `build-bun` jobs of #133628 asked for. All 85 jobs of the fork's build of this commit passed, its JavaScriptCore tests on Linux x64 and arm64 included. `test/js/bun/jsc/webkit-upgrade-191c6a4138.test.ts` covers oven-sh/WebKit#917: | Binary | Result | | --- | --- | | `main`'s CI build (92e819d, linux x64, the old pin) | 0 pass, 2 fail: the heap is 262,032 bytes (`WeakMap`) and 129,498 bytes (`WeakSet`) smaller after the eden collection | | Bun 1.4.4 (an engine from before the bug) | 2 pass | Both the same with `BUN_GARBAGE_COLLECTOR_LEVEL` 0, 1 and 2. It has not run on the new engine yet: CI is the first run. `bun test test/internal/source-lints/` passes (186). --- _Generated by [Claude Code](https://claude.ai/code/session_01YF5TQxMfo4rBWHbttbVAhn)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do? Consolidates the open pull requests on the CSS parser (`src/css`: parsing, minifying, lowering, printing, CSS modules) into one. 179 open pull requests were in scope. Each was judged by one rule: **the output must mean to a browser what the input meant.** Every claim was run through a build of `main`, a build with the change, lightningcss 1.33 and Chromium 156 (CSSOM, computed values, painted pixels for colors and gradients), and counted against 10,116 real stylesheets: the published CSS of about 190 popular npm packages, plus stylesheets from open source repositories. 107 land here, whole or in part, most of them cut down. The source diff is net negative. Of the 10,116, 9,509 are stylesheets that someone ships. The rest are test fixtures of other tools (521), Tailwind v4 source that needs its plugin (59) and SCSS, PostCSS or template source (27). | 9,509 shipped stylesheets | `main` | this branch | |---|---|---| | fail to build | 71 | 35 | | ... of those, valid CSS that Bun rejects | | 0 | | minified output reads differently in Chromium than the input | 552 | 2 | | CSS modules with a name that is referenced but not defined, or exported but not defined | 166 | 0 | | all 10,116 | `main` | this branch | |---|---|---| | fail to build | 192 | 141 | | minified output reads differently in Chromium than the input | 593 | 15 | | build on `main`, fail here | | 0 | | right on `main`, wrong here (minified, and at the default targets) | | 0 | | size of the minified output, no targets | 162.32 MB | 162.47 MB (+0.093%) | | size of the output at the default targets | 224.1 MB | 210.2 MB (-6.2%) | A syntax error is a build error, with one exception: the IE star hack `*zoom: 1` is skipped, because tachyons and handsontable ship it. The 35 shipped stylesheets that fail are invalid CSS: 28 have a selector that Chromium drops (semantic-ui-css, swiper, @ionic/core, Adobe Spectrum, xp.css), 2 an `@import` after other rules, and one each `@supports not X and Y`, a nested `@keyframes`, a stray comma, a selector list in `::slotted()`, `@media screen\0` (govuk-frontend). The 2 that read differently fold a `color-mix()` inside a gradient (see the known gaps). The growth of the minified output is rules and declarations that `main` deleted by mistake: 87 KB of rules, 45 KB of prefixed declarations that only browser targets can rule out. #### Popular CSS that `main` breaks today - normalize.css, modern-normalize, sanitize.css: `font-family: monospace, monospace` is collapsed to `monospace`, which changes the font size from 16px to 13px (oven-sh#46662). - CKEditor 5, Tabler, Carbon, VitePress, Ionic, PatternFly, Excalidraw: a rule of custom properties is deleted as a "duplicate" of a later rule with the same selector and other custom properties. 37 stylesheets lose 1,681 custom properties (oven-sh#33684). - Material Components, daisyUI, DocSearch, plyr: a `var()` size loses to its own fallback (oven-sh#42101). - nextra, daisyUI, CKEditor: declarations written after a nested rule are moved above it (oven-sh#45462). - antd 4, purecss, govuk-frontend: browser hacks in selector lists start to apply everywhere (oven-sh#42100). - foundation-sites, flatpickr, 98.css, blueprintjs, scalar, react-datepicker: the build fails on valid CSS (oven-sh#46756, oven-sh#38552, oven-sh#38534). tachyons, handsontable: it fails on the IE star hack (oven-sh#46751). - nodejs.org, Mantine: a CSS module exports classes that no rule defines, because the `@apply` or `@mixin` in them is moved out of the rule (oven-sh#46759). - hint.css and 42 more: `z-index: 1000001` and `1000000` become the same number (oven-sh#46715). - Adobe Spectrum: `--version: 5.0.1` prints as `5.1`. - A quarter of all stylesheets: minified output has a stray space in `align-items: center` (oven-sh#41993). #### Behavior changes to look at first 1. **`light-dark()` is kept as written at the default browser targets** (oven-sh#42009). The lowered form `var(--buncss-light, a) var(--buncss-dark, b)` is invalid unless a compiled `color-scheme` covers the element: 184 of the 215 real stylesheets that get it never define the variable, so every browser drops the declaration. The docs section on the polyfill is removed. 2. **`border` is not built from longhands** (oven-sh#42072): `border-width:0;border-style:solid;border-color:red` stays three declarations, because `border` also resets `border-image`. 305 real stylesheets grow by 30 KB in total; `bootstrap.min.css` by 113 bytes, 5 gzipped. No real stylesheet was found where the old merge shows, so this is the one to drop if size matters more. 14 existing expectations change. 3. **A CSS module no longer exports names that only occur inside `:global(...)`** (oven-sh#45487). 73 real modules lose 202 export keys, all of them names that no rule has. `import { g }` or `composes: g` of such a name becomes a build error. 4. **Container names and `@counter-style` names stay global in a CSS module** (oven-sh#45247 by @mikamikasuki). `main` hashes the `@container` name but not `container-name`, so a named query never matches. postcss-modules (Vite, css-loader, Next.js) keeps them global; lightningcss and esbuild hash both. Global keeps a container that is queried from another file or from JS working. 5. **Syntax errors are build errors, also two that `main` lets through**: a media query that stops short (`@media screen and {`, `@media not {`, `@import "a.css" screen and;`) and `@media (width <) foo {`. No stylesheet of the 10,116 has either. A condition that is valid by the grammar though no browser knows it, `( ... )` or `fn( ... )`, is kept as written (oven-sh#46756): `(min-width:0\0)`, `(not(hover))`. 6. **Declarations after a nested rule stay where they were written**, as a nested declarations rule (oven-sh#45462). Changes the output of 199 real stylesheets; in a fuzz of 2,850 nested programs compared by computed style, `main` was wrong in 1,056 and the pull request's code, before it was trimmed here, in none. 7. **`Bun.color`**: `color()` inputs are converted instead of returned as a string (`Bun.color("color(srgb 1 0 0)", "hex")` was `"color(srgb 1 0 0)"`); `currentColor`, system colors and `light-dark()` give `null` in the converting formats instead of the keyword as a string; a relative color or `color-mix()` outside sRGB is clipped, not gamut-mapped (`hsl(from oklch(100% .399 336.3) h s l)` is `#ff5bff`, was `#fff`). 8. The `:-moz-any()` and `:-webkit-any()` copies of rules with `:is()` are gone at the default targets (oven-sh#45945, oven-sh#42029). The `:-webkit-any()` copy outranked the original rule: `.title{color:blue} :is(h1,h2,h3){color:red}` painted red. 5.8% of all default-target output was the `:-moz-any()` copies. 9. `Property` grows from 216 to 224 bytes: `Url` stores its text as a slice so that two `url()` values can be compared (oven-sh#46895). The pull request packed a pointer and a `u32` behind `unsafe` to avoid that. #### Not in this PR, needs a decision - **Raise the default browser targets (oven-sh#40369) and add a `cssTarget` option (oven-sh#40368).** Today: Chrome 80, Edge 80, Firefox 78, Safari 14, Opera 67, and no public way to change them. At Vite 8's floor the output of the 9,924 stylesheets is 15.3% smaller (3.9% gzipped). About a dozen fixes here only matter at today's floor, and code for older targets is unreachable without the option. - Vendor prefixes for `user-select`, `appearance`, `hyphens` and five more (oven-sh#45814): 1,116 real stylesheets lack a prefix they need at the default targets. And for `backdrop-filter` (oven-sh#44648): 173. - Clip instead of gamut-map in the sRGB fallback of a plain wide-gamut color: deletes about 90 lines, changes every pinned fallback hex. If the mapping stays, oven-sh#44647 by @oddharsh makes it closer to what browsers paint (colors off by 2/255 or more: 168 to 110 of 3,170 real ones). #### Changes **Rules: duplicates, merging, nesting, fallbacks** - The `:-moz-any()` copy of every rule with `:is()` or `:not(a, b)` is gone: Firefox 78 has `:is()` (oven-sh#45945). 1,164 real stylesheets - Keep declarations written after a nested rule in place with a nested declarations rule (oven-sh#45462). 379 real stylesheets - With nesting compiled away, an unknown at-rule inside a style rule was moved out of it: `.wrapper{@apply flex}` printed a bare `@apply flex;` and the rule was gone, so a CSS module exported a class that no rule defines. It now stays in a block under the compiled selector. 72 real modules with Tailwind's `@apply`, 25 with Mantine's `@mixin` (oven-sh#46759). 316 real stylesheets - Keep !important in the rules generated for logical properties and color-scheme (oven-sh#38597). 311 real stylesheets - With browser targets, remove an earlier duplicate rule only when its declaration block is equal (oven-sh#46861). 171 real stylesheets - Compare custom property names in `PropertyId` equality and importance in duplicate-rule detection (oven-sh#33684). 40 real stylesheets - Keep same-condition @supports rules and empty named @layer blocks (oven-sh#46733). 31 real stylesheets - Keep target-added vendor prefix passes on cloned and re-minified rules, and never print a bare `:full-screen` (oven-sh#43339). 12 real stylesheets - Keep the selectors on rules added for color fallbacks and logical properties (oven-sh#43240). 3 real stylesheets - A later declaration of a property drops the `@supports` color copies of the earlier one, which printed after the rule and won (oven-sh#45560). 2 real stylesheets - Do not merge style rules in a stylesheet that holds `revert-rule` (oven-sh#46953). 1 real stylesheets - `@import ... supports(-webkit-mask: none)` kept its prefix (oven-sh#38589) **Properties and shorthands** - Print align-items:center without a stray space when no safe/unsafe keyword is given (oven-sh#41993). 3,317 real stylesheets - Emit the physical longhands instead of the inset shorthand for targets without it (oven-sh#38583). 1,452 real stylesheets - Keep the stops of an upward gradient as written, and drop a `50%` hint only at the midpoint (oven-sh#46790). 104 real stylesheets - Emit unparsed border-block-* declarations when compiling logical borders (oven-sh#38582). 89 real stylesheets - Keep repeated font families and dedupe only the inserted system-ui fallbacks (oven-sh#46662). 88 real stylesheets - A `width` / `height` / `min-*` / `max-*` with `var()` or `env()` was printed before the plain fallback written above it, so the fallback won (oven-sh#42101). 26 real stylesheets - Keep box-orient and box-direction when flex-direction or flex-flow follows (oven-sh#45672). 16 real stylesheets - Write held prefixed background declarations at every flush (oven-sh#46882). 6 real stylesheets - Derive equality for Size2D and BorderRadius, and keep every prefix in the border-radius handler (oven-sh#45665). 2 real stylesheets - Print `none` instead of an empty border-image, mask-border or mask-box-image value (oven-sh#45143). 2 real stylesheets - Keep font/transition sub-properties on the side of the shorthand they were written on (oven-sh#41990). 2 real stylesheets - CSS-wide keywords are not names: `font:12px Arial;font-family:inherit` printed `font:12px inherit`, `color-scheme:inherit` and `color-scheme:auto` printed `normal`, `transition-property:unset` was folded into `transition` (oven-sh#42022). 2 real stylesheets - Only fold font longhands into `font` when the rule declared a `font` (oven-sh#45459) - Keep a font family name quoted when its identifiers do not read back as the same name (oven-sh#46606) - Stop rewriting valid positions into ones browsers reject (oven-sh#45197) - `border` is no longer built from longhands unless the rule also sets what `border` resets (`border-image`), and an author's `border-image` reset is no longer dropped. `transition` is likewise only written where the rule wrote one, because it resets `transition-behavior` (oven-sh#42072) - `border; border-image: <gradient>; border-image-slice: 1` lost the gradient in every default build (oven-sh#42018) - A prefixed `linear-gradient` keeps its direction, written or generated: the prefixed syntax names the starting side and measures angles differently (oven-sh#46697) **At-rules** - Print @font-face format() keywords as strings (oven-sh#37979). 439 real stylesheets - Print @supports `not` conditions without a leading space (oven-sh#38559). 88 real stylesheets - Reset after the number token of a unicode-range (oven-sh#45835). 33 real stylesheets - Keep `<general-enclosed>` media and container conditions as written (oven-sh#46756). 24 real stylesheets - A strict range such as `(width < 768px)` lowered for old targets still matched at exactly 768px. The step is now `.02px`. The `not all and` form for other units is not taken (oven-sh#46806). 18 real stylesheets - Keep scanning an @supports condition after a nested block (oven-sh#38552). 9 real stylesheets - Every `@counter-style name` printed as `@counter-stylename`, which browsers drop (also oven-sh#46692). In a CSS module the name now stays global, like the `list-style` that refers to it (oven-sh#33679). 9 real stylesheets - Keep a keyframe whose block has an invalid item (oven-sh#46288). 2 real stylesheets - Read an @Property initial-value on the stylesheet's parser, keep an unmatched value as written (oven-sh#45806). 2 real stylesheets - `@media {}` is `all`. A media query that stops after `and`, `or` or `not` (`@media screen and {`) is a build error; `main` printed `not all` or dropped the condition, silently. The `@layer` and type selector parts are not taken (oven-sh#46869). 2 real stylesheets - Accept a name-only `@container` and a `style()` feature with no value (oven-sh#46689). 1 real stylesheets - Parse `<timeline-range-name> <percentage>` keyframe selectors (oven-sh#45652) - Allow `or` inside parentheses after a media type (oven-sh#45088) - Print the `=` comparison in media and container queries as `=` (oven-sh#45071) - Parse the @font-palette-values font-family descriptor as a list of family names (oven-sh#45484) **Selectors** - Never wrap a selector list in :is() (oven-sh#46787). 414 real stylesheets - Stop emitting :-webkit-any() copies of :is() that outrank the original rule (oven-sh#42029). 373 real stylesheets - With browser targets, a selector list with a vendor-prefixed member was split, which revives hacks that rely on other engines dropping the whole list: `.s *::-ms-backdrop,.s{flex:auto}` applied everywhere. Rewritten; the list is printed as written (oven-sh#42100). 42 real stylesheets - Print the vendor prefix on downleveled :any-link, :read-only, :read-write, :placeholder-shown and :autofill (oven-sh#42131). 12 real stylesheets - Fix nesting lowering of a `&` that does not lead its selector (oven-sh#43047). 8 real stylesheets - `@scope`: `&` and implicit selectors compile to `:where(:scope)`, relative selectors and bare declarations parse (oven-sh#43165). 7 real stylesheets - Only reject bare :global / :local pseudo-classes in CSS modules (oven-sh#38534). 5 real stylesheets - Stop truncating `@scope to (...)` and stop replacing `&` in the scope-end selector (oven-sh#43146). 2 real stylesheets - `[xl|href$=".pdf"]` lost its namespace prefix (oven-sh#45159). 2 real stylesheets - Match An+B keywords exactly and take only `+` as the leading sign (oven-sh#42953). 1 real stylesheets - Parse ::details-content and ::picker() like ::part() (oven-sh#45504). 1 real stylesheets - Parse :window-inactive as an ordinary pseudo-class, and accept it after ::selection (oven-sh#45815) - Print :nth-col(1) and :nth-last-col(1) in functional form (oven-sh#41575) - `li::before::marker` failed the build (oven-sh#45810) - `&` inside `:nth-child(... of S)` is resolved when nesting is compiled away. The second selector serializer (326 lines) is deleted (oven-sh#43024) **Colors and `Bun.color`** - Keep light-dark() as written under the default browser targets (oven-sh#42009). 227 real stylesheets - Number and percentage channels parse in `lab()`, `lch()`, `oklab()`, `oklch()`, `hsl()` and `hwb()`; the hue of `lch` / `oklch` is no longer scaled by alpha (oven-sh#45084). 130 real stylesheets - `Bun.color` returns `null` for `currentColor`, system colors and `light-dark()` in the converting formats, instead of the CSS keyword as a string (oven-sh#33047). 130 real stylesheets - Relative colors: channel keywords are on the scale of their function (`rgb(from oven-sh#369 calc(r + 10) g b)`), the origin's alpha is kept, a relative color kept as written gets no `@supports` copies, and a result outside sRGB is not folded (oven-sh#45362). 31 real stylesheets - Keep rotate: 0deg out of none, and keep the origin alpha in relative colors (oven-sh#38486). 13 real stylesheets - Compute the D50 reference white in f32 so greys convert to lab(l 0 0) like lightningcss (oven-sh#38535). 7 real stylesheets - Return a missing hue for a grey from conversions into lch, oklch, hsl and hwb (oven-sh#45201). 5 real stylesheets - Fix color(a98-rgb), srgb-linear relative colors and color-mix(in xyz-d50) (oven-sh#38487). 4 real stylesheets - Match named colors without regard to ASCII case (oven-sh#45092). 3 real stylesheets - Fold min(), max() and clamp() over plain numbers (oven-sh#39515) - `calc(l - .08)` in a percentage channel of a relative color became NaN and printed `#000` (also oven-sh#45970) (oven-sh#45971) - Bun.color: convert color() inputs instead of returning them as a CSS string (oven-sh#38488) - A `color-mix(in srgb, ...)` result outside sRGB prints `color(srgb ...)` instead of a gamut-mapped hex (oven-sh#39269) **`calc()` and numbers** - Rounding to 6 significant digits ate into the integer part: `z-index:2147483647` became `2147480000`, `max-width:9999999px` became `10000000px`, `--x:1234567.5` became `1234570`. It now only cuts a fraction short: a value kept as tokens prints its source text, a typed value prints every integer digit and at least one digit of the fraction (oven-sh#46715). 43 real stylesheets - A number without a unit is no longer repaired into `px` (`height:1.5715` is dropped by a standards-mode browser, so it prints as written), `calc(-1 * 0)` is not a length, and on a property of the range [0,∞] (sizes, border radii, padding, border and outline widths, gap, `flex-basis`, `font-size`) a `calc()` that folds below zero prints `0` instead of an invalid negative literal, and a negative literal no longer replaces the valid declaration before it (`width:10px;width:-5px` printed `width:-5px`) (oven-sh#42104). 24 real stylesheets - Print numbers from the f32's shortest digits, not its f64 expansion (oven-sh#38565). 17 real stylesheets - Print negative numbers below 1 with a single minus sign (oven-sh#38516) - Keep math functions over two different length units as written (oven-sh#38489) - Fold rem() with a truncated quotient, so it keeps the sign of the dividend (oven-sh#32286) - A one-argument `min()` / `max()` around a sum printed without any function (`width:100% - 20px`) (oven-sh#46145) - `round()` breaks ties upward and handles a negative step as browsers do (oven-sh#45249) **CSS modules** - A CSS module exports its `@keyframes` names (`styles.spin`). Hashing is unchanged (oven-sh#46720). 145 real stylesheets - A name inside `:global(...)` prints plain but was exported as `name_<hash>`, a class no rule has (also oven-sh#38573) (oven-sh#45487). 90 real stylesheets - Keep container names global in CSS modules (oven-sh#45247). 6 real stylesheets - Tag the keyframes name when an `animation` value falls back to raw tokens (oven-sh#46034). 6 real stylesheets - In a plain stylesheet `composes: a from "./b.css"` was treated as an import: the file was inlined and an absolute path printed (oven-sh#45961). 4 real stylesheets - Drop a rejected composes instead of failing the file at print time (oven-sh#38538). 1 real stylesheets - Keep every rule's custom properties for the composes conflict check (oven-sh#38523) - Keep composes working after a nested rule in the same block (oven-sh#38520) - `IdentOrRef::hash` read 2 of 16 bytes, so in release builds every class of a module hashed alike (oven-sh#30784) **Tokens, `url()`, diagnostics** - The block skip stack no longer allocates on every block exit (oven-sh#39533) - Report the diagnostic's line without its line breaks in position.lineText (oven-sh#38342). 192 real stylesheets - Compare `url()` values by their text (oven-sh#46895). 26 real stylesheets - Values kept as tokens keep the blanks they were written with instead of having them re-invented at print time: `attr(x type(<number>))` printed `type(< number> )`, which browsers drop. Minified output still drops the blanks that cannot matter (oven-sh#45577). 19 real stylesheets - The IE star hack (`*zoom:1`) failed the whole build. The declaration is skipped, as every browser does. It is the one syntax error that builds: tachyons, handsontable and jasmine ship it (oven-sh#46751). 8 real stylesheets - The path of a copied asset spliced into `url()` is always quoted, so a file name with a space or a parenthesis cannot end the `url()` (oven-sh#41557). 7 real stylesheets - Move the skip loop of parse_until_before out of the generic body (oven-sh#39520) - Remove dead code from bun_css: the unused `impl Default for Property` (oven-sh#45618) - Default the at-rule and qualified-rule parser traits, drop the url_to_css shim (oven-sh#31999) **Speed** Four inputs that took quadratic or exponential time, or produced output out of proportion, are now linear, with no new limit and no extra pass (oven-sh#31639, oven-sh#42243, oven-sh#38531, oven-sh#43421). **Tests only** - `css-fuzz.test.ts` never ran in CI and timed out on debug builds; it is a deterministic table now (oven-sh#38500) - `nth-anplusb-ident.test.ts` counts heaps in a child process (oven-sh#46331) - `color.test.ts` checks the ansi-16 sweep with one `expect()` (oven-sh#42824) - Not CSS, but it failed this PR's CI and fails on `main` too: `ws-proxy.test.ts` on Windows, "a message sent during a TLS renegotiation arrives after it". The helper server now renegotiates from the callback of the frame's write (oven-sh#46170) **No pull request covered these** - `--v: 5.0.1` printed as `5.1`, `--w: 1.0.0` as `10`, `--x: 1+2` as `12`, `2n+1` as `2n1`: a number token that touches another keeps its source text. - `color-scheme: auto` printed as `normal`, and `color-scheme: dark light` as `light dark`. - A bare `:global` or `:local` in a CSS module sets the scope of the names after it, up to the end of the selector, as in css-loader. It was a build error. - The scrollbar states (`a:horizontal`, `::-webkit-scrollbar{&:horizontal{}}`) are valid in a plain compound, as in WebKit and Blink. - Without browser targets, `-webkit-box-align`, `-ms-flex-align` and the like were dropped next to their standard form (flatpickr, foundation-sites, materialize-css: 28 stylesheets), and `-webkit-fill-available` before `-moz-available` (beercss). - `(-o-min-device-pixel-ratio: 2/1)` printed as `2`. - `sin()`, `cos()` and `tan()` are exact at multiples of 90deg: `calc(100px * cos(90deg))` printed `-.00000437114px`. - A `display-p3` color at targets that have P3 but not the other `color()` spaces (Safari 10.1 to 14) got itself as a fallback. - `width: rem(-18px, 5px)` folds to a negative literal once `rem()` is right; it prints `0`. - Consecutive nested declarations rules, which arise when same-query `@media` blocks in one rule merge, are folded into one and count against the existing selector expansion limit as the `&{}` rules they replace did. #### Known and not fixed here - Four longhands are still folded into a `transition` the author did not write where the targets add a prefixed copy of a property (`mask-image` at Chrome 80): only the shorthand gives the copy its own times. - A later `background`, `box-shadow` or `border-*` declaration still loses to the `@supports` copy of an earlier `var()` value of the same property with a wide-gamut color. Same on `main`; oven-sh#45560 is taken for the properties of the fallback handler and for unparsed values only. - A merged rule prints the unprefixed name in the passes of the rule it absorbed: `.d::-webkit-input-placeholder{}.d::-moz-placeholder{}` prints `::placeholder` for the second (oven-sh#42106, not taken: it breaks the merge of prefix-equivalent rules). - The `border`, `outline` and `font` shorthands, `line-height`, `scroll-padding`, `background-size` and a few more still print a `calc()` that folds below zero as a negative literal. No real stylesheet has one. - `all` is not a property Bun knows, so `a{margin-top:1px;all:unset}` prints `all` first. Same on `main`. - A selector kept as raw tokens loses the blank before `*` when minified: `--sel: .dark *` prints `.dark*`. Same on `main`. - `width:calc(0% + 12px)` folds to `12px`, which lays out differently in a table cell. Same on `main`. - The `:dir()` / `:lang()` rules of a lowered logical property print before the nested rules of their rule, so a logical declaration written after a nested rule of equal specificity still loses to it. Same on `main`. - A fraction is still rounded to 6 significant digits (`12345.678` prints `12345.7`, `33.3333333333%` prints `33.3333%`). Keeping every digit would add up to 282 KB over the corpus. - `@keyframes "quoted"` is hashed in a module but not exported. - In a module, a keyframes name inside a `var()` fallback is not hashed: `animation: var(--enter, fade 1s)`. Same on `main`. - A `color-mix()` or relative color folded to a hex inside a gradient changes the interpolation from Oklab to sRGB (VS Code, clawhub). Printing `color(srgb ..)` instead needs a fallback for `mask-image`, `list-style-image` and custom properties at the default targets, which have none. - A comma list in `@container` still fails the build. - After a bare `:global`, names that are hashed at print time (`::view-transition-old(foo)`) are still hashed; inside `:global(...)` they are not. #### Pull requests in scope that do not land here - Duplicates of a pull request that lands here: oven-sh#36961 (of oven-sh#37979), oven-sh#38478 (of oven-sh#46662), oven-sh#38480 (of oven-sh#46733), oven-sh#38482 (of oven-sh#46715), oven-sh#38491 (of oven-sh#38500), oven-sh#38508 (of oven-sh#45201), oven-sh#38513 (of oven-sh#45362), oven-sh#38573 (of oven-sh#45487), oven-sh#38640 (of oven-sh#38500), oven-sh#45525 (of oven-sh#45247), oven-sh#45589 (of oven-sh#46756), oven-sh#45970 (of oven-sh#45971) - Already fixed on `main`: oven-sh#38674 - Right, but for input nobody writes (0 of 10,116 stylesheets, and not new syntax): oven-sh#31916, oven-sh#38526, oven-sh#38592, oven-sh#39349, oven-sh#41139, oven-sh#41801, oven-sh#42042, oven-sh#42484, oven-sh#43078, oven-sh#45604, oven-sh#45908, oven-sh#46086, oven-sh#46221, oven-sh#46224, oven-sh#46273, oven-sh#46694, oven-sh#46707, oven-sh#46737, oven-sh#46755, oven-sh#46816, oven-sh#46870, oven-sh#46934 - Right or harmless, but not worth their code, their bigger output or the behavior they change: oven-sh#30905, oven-sh#31924, oven-sh#33332, oven-sh#34426, oven-sh#36225, oven-sh#38006, oven-sh#38512, oven-sh#38547, oven-sh#38551, oven-sh#38561, oven-sh#38572, oven-sh#38576, oven-sh#38607, oven-sh#38623, oven-sh#38667, oven-sh#38692, oven-sh#42106, oven-sh#42912, oven-sh#42918, oven-sh#43407, oven-sh#43434, oven-sh#44700, oven-sh#45442, oven-sh#45546, oven-sh#45747, oven-sh#45759, oven-sh#45825, oven-sh#45946, oven-sh#46061, oven-sh#46127, oven-sh#46206 - Need a decision (see above), stay open: oven-sh#40368, oven-sh#40369, oven-sh#44647, oven-sh#44648, oven-sh#45814 - Stays open: oven-sh#45787 Out of scope: CSS linking and chunking in the bundler, the dev server, docs-only pull requests, and repository-wide sweeps that touch `src/css` in passing. Thanks to @mikamikasuki (oven-sh#45247), @alii (oven-sh#39520, oven-sh#31999) and @oddharsh, whose measurements on oven-sh#40369 and oven-sh#41990 were used. Fixes oven-sh#16727. Fixes oven-sh#24770. Fixes oven-sh#30772. Fixes oven-sh#44704. Fixes oven-sh#45089. Fixes oven-sh#45237. Fixes oven-sh#46872. ### How did you verify your code works? - The 35 CSS test files pass on a debug build, also with `FORCE_COLOR=1`. 600+ of the tests in this branch fail on a build of `main`. An earlier head of this branch passed all 179 CI jobs that ran. - The reproduction of each issue listed above was run on `main` (wrong) and on this branch (right). - Differential run over the 10,116 stylesheets, `main` against this branch, minified without targets and through `Bun.build` at the default targets: no stylesheet that builds on `main` fails; Chromium 156 compares the output with the input rule by rule (CSSOM, then computed values on a sized element in a standards-mode document, colors as painted pixels in sRGB and P3). The comparison has 195 control pairs that must read equal or different. For stylesheets that read differently on both builds, mostly because of lowering, the sets of differences were compared; the three new ones are two minifier fixtures that are wrong on both and one comparison artifact. - CSS modules are checked for consistency instead, since hashed names defeat the comparison: every reference points at a name the output defines, every export exists. - `Bun.color`: 59 inputs in 13 formats on both builds; every changed cell was read.
…e --aot` (oven-sh#45631) ```sh bun build --compile --aot ./app.ts ./app ``` This adds `--aot` to `bun build --compile`. It compiles TypeScript and JavaScript to machine code at build time. The executable starts as machine code and has no JIT in it. `bun compile ./app.ts` is short for the same command. It's still JavaScript. npm packages, CommonJS, `eval`, proxies and `any` all work. There's no new syntax and nothing to annotate. The new part is that the compiler reads your TypeScript types and uses them. ```ts class Vec { constructor( public x: number, public y: number, ) {} } function dot(a: Vec, b: Vec): number { return a.x * b.x + a.y * b.y; } ``` A `Vec` is a 16-byte header and two raw doubles. Where the compiler can prove that only `Vec`s reach `a.x`, the read is one load. No hidden-class check, no inline cache, no boxed number. Where it can't prove that, it tests once and keeps a slow path. It's experimental, and it [loses in places](#where-it-loses). The engine half is oven-sh/WebKit#757. The docs start at `docs/bundler/aot.mdx`. ## The TypeScript compiler, compiled TypeScript 6 is written in TypeScript. TypeScript 7 is the TypeScript team's port of it to Go. We built the `v6.0.3` tag from its own source with `--aot` and type checked [effect](https://github.com/Effect-TS/effect) with it, 370,839 lines. Not a line of TypeScript's source was changed. | | wall, s | peak RSS, MB | CPU, s | | --- | ---: | ---: | ---: | | TypeScript 7 (Go) | 2.68 | 1,256 | 10.8 | | **TypeScript 6, `--aot`** | **4.25** | **963** | **5.3** | | TypeScript 7 (Go), `--singleThreaded` | 5.08 | 933 | 6.2 | | TypeScript 6 on Bun 1.4.3 | 14.00 | 2,581 | 29.2 | `tsc -p` on effect's `packages/effect`, 0 errors in every row. Linux x64, AMD EPYC 9R14, pinned to 16 cores that nothing else uses. LTO build, median of five runs, 2026-10-24. The two TypeScript 6 rows are the same compiler at the same tag. Compiled ahead of time it's 3.3x faster than on Bun 1.4.3, in 2.7x less memory and 5.5x less CPU time. Go checks in parallel, which TypeScript 6 doesn't, and that wins on the clock. Take the parallelism away with `--singleThreaded` and the original TypeScript finishes first, on less CPU time and about the same memory. Both still collect garbage on other threads in that comparison. <details> <summary>How to run it yourself</summary> Bun 1.4.3, TypeScript 7.0.2 (`@typescript/typescript-linux-x64`), TypeScript 6.0.3 (tag `v6.0.3`, sources and `lib/` unmodified), effect at `8b0eac6a`, `@types/node` 26.6.4. ```sh git clone --depth 1 -b v6.0.3 https://github.com/microsoft/TypeScript git clone https://github.com/Effect-TS/effect && git -C effect checkout 8b0eac6ac46a689a7208500e2ccf9e3b80fc3898 npm install typescript@7.0.2 @types/node@26.6.4 # --aot type checks what it compiles, and a bare checkout lacks two of TypeScript's # dev dependencies (@types/node, source-map-support). This supplies them. echo 'declare module "source-map-support";' > missing.d.ts cat > aot-tsconfig.json <<X { "extends": "$PWD/TypeScript/src/tsconfig-base.json", "compilerOptions": { "rootDir": "$PWD/TypeScript/src", "types": ["node"], "typeRoots": ["$PWD/node_modules/@types"], "composite": false, "declaration": false, "declarationMap": false, "emitDeclarationOnly": false, "isolatedDeclarations": false, "noEmit": true }, "files": ["$PWD/TypeScript/src/tsc/tsc.ts", "./missing.d.ts"] } X bun build --compile --aot --production --format=esm --tsconfig-override=aot-tsconfig.json \ --define "__filename=\"$PWD/TypeScript/lib/tsc.js\"" --define "__dirname=\"$PWD/TypeScript/lib\"" \ TypeScript/src/tsc/tsc.ts --outfile tsc-aot go=node_modules/@typescript/typescript-linux-x64/lib/tsc run="taskset -c 0-15 /usr/bin/time -v" $run $go -p tsconfig.json --pretty false $run ./tsc-aot -p tsconfig.json --pretty false $run $go -p tsconfig.json --pretty false --singleThreaded $run bun TypeScript/lib/tsc.js -p tsconfig.json --pretty false ``` `tsconfig.json` is effect's own options for `packages/effect`, with absolute paths: ```json { "include": ["<here>/effect/packages/effect/src"], "compilerOptions": { "noEmit": true, "incremental": false, "composite": false, "rootDir": "<here>/effect/packages/effect/src", "target": "ES2022", "module": "NodeNext", "moduleDetection": "force", "verbatimModuleSyntax": true, "allowJs": false, "rewriteRelativeImportExtensions": true, "erasableSyntaxOnly": true, "strict": true, "exactOptionalPropertyTypes": true, "noUnusedLocals": true, "noUnusedParameters": true, "noImplicitOverride": true, "noFallthroughCasesInSwitch": true, "stripInternal": false, "skipLibCheck": true, "noErrorTruncation": true, "jsx": "react-jsx", "types": ["node"], "typeRoots": ["<here>/node_modules/@types"], "preserveSymlinks": true } } ``` </details> ## A real CLI Claude Code's CLI ships as a `bun build --compile` executable. This is the released build against the same app built with `--aot`, in a scripted session: start, first prompt, five turns, with the API stubbed out locally. | | released | `--aot` | | --- | ---: | ---: | | time to first prompt, ms | 318 | **141** | | peak private memory, MB | 185.4 | **44.7** | | private memory at the first prompt, MB | 112.0 | **28.7** | | private memory at the end, MB | 150.1 | **37.5** | | peak RSS, MB | 308.5 | **180.8** | | CPU time, ms | 1,542 | **512** | | instructions, millions | 5,530 | **1,117** | Linux x64, AMD EPYC 9R14, 16 cores that nothing else uses. LTO build, default flags, median of 16 sessions per column, alternating, 2026-10-24. CPU time and instructions count all threads. The first prompt is up in under half the time, on a third of the CPU time and a quarter of the private memory. ## Servers One small API, written seven times (`test/aot/servers/app/api-*.ts`): by hand on `Bun.serve` and on `node:http`, and on Hono, Elysia, Express, Fastify and Koa. It has three middlewares (a request id, response headers, a bearer token), route parameters, a query string, JSON in and out, an error handler and a 404. Each server is built twice by the same `bun`: with `--aot`, and with plain `--compile`, which runs it on the JIT. Same runtime, same built-in modules, same native code. Only the compiler differs. Requests per second in the first second after the process starts: | | `--aot` | JIT | ratio | | --- | ---: | ---: | ---: | | Koa | **33,109** | 17,180 | 1.93x | | Express | **46,562** | 24,181 | 1.93x | | Fastify | **57,976** | 37,822 | 1.53x | | `node:http` | **69,085** | 46,693 | 1.48x | | Hono | **58,978** | 48,029 | 1.23x | | `Bun.serve` | **84,599** | 77,867 | 1.09x | | Elysia | **63,556** | 61,560 | 1.03x | Requests per second once the JIT is warm, over requests 100,001 to 400,000 of the same process: | | `--aot` | JIT | difference | | --- | ---: | ---: | ---: | | Hono | 59,737 | 58,151 | +2.7% | | `Bun.serve` | 85,553 | 83,490 | +2.5% | | `node:http` | 70,205 | 70,383 | -0.3% | | Express | 47,935 | **51,587** | -7.1% | | Elysia | 65,110 | **75,268** | -13.5% | | Fastify | 59,656 | **71,796** | -16.9% | | Koa | 34,194 | **44,362** | -22.9% | The same program built twice differs by up to 1.9%, so we call anything under 3% level. That's the top three. The other four lose. Counting every request since the process started, the JIT has caught up within 2 seconds on Elysia and within 5 on Fastify and Koa. Express is still ahead at 5 seconds. Startup and memory: | | first response, ms, `--aot` | first response, ms, JIT | peak RSS, MB, `--aot` | peak RSS, MB, JIT | | --- | ---: | ---: | ---: | ---: | | `Bun.serve` | 8.3 | **7.7** | **32.1** | 45.7 | | Hono | **9.3** | 11.5 | **34.4** | 51.8 | | Elysia | **11.5** | 27.5 | **40.8** | 56.6 | | `node:http` | **11.5** | 21.4 | **43.8** | 61.8 | | Fastify | **17.0** | 63.4 | **51.1** | 73.1 | | Express | **18.3** | 50.4 | **54.4** | 85.9 | | Koa | **20.1** | 66.7 | **57.2** | 85.3 | All three tables: Linux x64, AMD EPYC 9R14, each server pinned to one core and its load generator to another, both columns of a row at the same time. LTO build, 64 connections, six routes in turn, every response compared byte for byte, median of 7 starts, 2026-10-24. Memory is read at request 400,000. `test/aot/servers/measure/requests_a_second.py`. There's no source to parse, no bytecode, no JIT-compiled code and no profiling data to keep around. The machine code is file-backed pages of the executable, so the kernel shares it between processes. Private memory is 13 to 25 MB compiled and 15 to 47 MB on the JIT. The executables are 69 to 82 MB, against 141 to 144 MB. At a fixed 5,000 requests per second, the 99th percentile of latency is 54 to 91 µs compiled and 77 to 477 µs on the JIT. The medians are 21 to 40 µs and 21 to 34 µs. ## Where it loses A warm JIT still wins on untyped npm code. That's the second servers table. Koa, Fastify and Express are untyped JavaScript in many small packages. A JIT learns which objects reach a property access by watching the code run. The compiler has to prove it at build time, and where it can't, the access is a lookup by name. [Full types](#full-types) are the way out we're betting on. Code that a package generates at runtime is interpreted, for as long as the program runs. zod 4, ajv, fast-json-stringify and Elysia generate their hot code with `new Function`. That's most of Elysia's loss above. For validation there's a way that needs no generated code: [`JSON.parse<T>` and `validate<T>`](#what-it-does-with-types). Old, untyped JavaScript is slower than on the JIT: constructor functions, class helpers, asm.js output, arithmetic on whatever comes out of a plain array. A JIT watches `a + b | 0`, sees two int32s every time and compiles an add. The compiler has to prove that they're int32s, and where it can't, the `+` is a call to a generic add. JetStream2, the benchmark that JavaScriptCore's JIT is tuned on, is mostly code like this, and most of its tests are slower compiled. Everyday idioms are mixed. `test/aot/perf` has 1,814 of them. Each is compiled twice, as written and with its types erased, and the slower of the two is compared with the same `bun`'s warm JIT, by instructions. | | ahead | within 5% | behind | median ratio | | --- | ---: | ---: | ---: | ---: | | each idiom as a small program | 916 | 223 | 675 | 0.94 | | compiled as in a very large program | 467 | 118 | 1,229 | 1.44 | `bun test/aot/perf/idioms-against-the-jit.ts`, and again with `--in-a-very-large-program`. Linux x64, AMD EPYC 9R14, 2026-10-24. User-mode instructions per iteration, with startup and warm-up subtracted. In the first row over a third are behind, 180 of them by more than 2x. And that row flatters the compiler. Each idiom is a program of its own, so its loop counts as hot and the idiom is inlined into it. A very large program can't have everything hot, and with no profile the compiler has to guess from loops and the call graph. The second row compiles the same idioms the way they'd be compiled there, and calls each through a function that's found at runtime, on the JIT too. Two thirds are behind, 687 by more than 2x. Builds are slow. The build type checks the whole program and compiles every function that can be reached. There's no cache between builds, so a one-line change builds everything again. A large application takes tens of seconds and gigabytes of memory. Executables are big. A small server is 69 MB. That's half of what `--compile` makes, and it's still 69 MB. [`--trim`](#smaller-executables) helps. `bun` itself [grows by about 30 MB](#what-plain-bun-pays). Go and Rust still win on small programs. A hello world in either starts sooner, holds a few megabytes and is a file of a few megabytes. Ours carries Bun's runtime. ## Why When a JavaScript tool gets too slow, the usual fix is to port it to Go or Rust. esbuild, SWC, Biome, oxc, Rolldown, Turbopack and now TypeScript itself went that way. It works, but it takes years, and afterwards most of the tool's users can't read or patch it. JavaScript engines were designed for browsers. Code arrives as text from somewhere you don't trust, has to start right away, and can do anything. So the engine interprets it, watches which types show up, guesses, compiles the guess, and throws the compiled code away when the guess was wrong. All of that happens on the machine that runs the program, every time it starts. A server or a CLI isn't in that position. You have all the source before you ship it. If it's TypeScript, you've also written down what everything is. Then the build erases that, and every process spends its first seconds working it out again. `--aot` keeps the types and does the work once, at build time. ## What it does with types TypeScript is unsound on purpose. `any`, `as`, `!`, `JSON.parse` and `.d.ts` files all let you call something a `number` when it isn't. So the compiler reads a type in two ways. For what your program makes, the type is a fact. `new Vec(1, 2)`, or an object literal of a declared type, gets the layout of that type. For what your program is given, the type is a prediction. An argument from a caller that the build can't see, or the result of `JSON.parse`, is tested once where it comes in. A value that fails the test takes a slow path. ```ts function double(n: number): number { return n * 2; } const body: any = JSON.parse('{"visits":"3"}'); console.log(double(body.visits)); ``` ``` $ bun app.ts 6 $ bun build --compile --aot app.ts && ./app 6 ``` `double` takes a raw number, so it can't take the string `"3"`. A guard at its entry sees the string and sends the call to a second, slow version of `double`, which multiplies the way plain JavaScript does. The string stays a string. The next call with a number takes the fast version again. The rules we hold the compiler to: 1. A program that runs under `bun run` does the same thing compiled. 2. Nothing throws at runtime because of a type. `as`, `any` and `!` never convert a value and never throw. 3. New errors show up at build time, with a message. The exceptions are listed under [What changes](#what-changes). JavaScript files are compiled the same way. A class has a fixed layout there too, and a flow analysis over the whole program follows objects from where they're created to where they're used, with or without annotations. Every property access ends up in one of three states: | A property access is | When | It costs | | --- | --- | --- | | free | The compiler proved which objects get here: from the types, from following every object through the program, or from a test earlier in the function | One load or store | | tested | A value that nobody checked can get in | A shape test, then the load. If the test fails, the slow path | | by name | Nothing is known | A lookup at runtime, through a cache | The build prints the split, and `--explain` lists what's behind the slow ones, worst first, with a fix for each. ``` 92.4% of property accesses are one load: 1,204 free, 61 tested, 38 by name (1,303) ``` An object with a fixed layout is still an object. `Object.keys`, `JSON.stringify`, spread and `for...in` see it like any other. You can add properties that the class didn't declare, delete the ones it did, and store a string in a `number` field through an `any`. That costs speed and nothing else. `JSON.parse<T>(text)` (`JSON` from `"bun"`) and `validate<T>(value)` (from `"bun:bundle"`) check data against a type you've already declared, and make objects with the layout of `T`. These two do throw. That's what they're for. `import type { int32 } from "bun"` is a 4-byte integer field. To TypeScript and under `bun run` it's a `number`. The build has to prove every store into one, and stops with an error where it can't. ## What changes Each of these is there because the compiler can't generate good code without it. `docs/bundler/aot-compatibility.mdx` has the smaller differences and a list of packages that are known not to work. | Change | What it buys | | --- | --- | | Type errors stop the build. They're TypeScript 7's errors, from the checker behind `bun check` | The types that the compiler starts from agree with each other | | Existing members of built-in objects can't be replaced. `Array.prototype.push = fn` throws a `TypeError`, in sloppy mode too. Adding is fine: `Array.prototype.last = fn`. `Object.prototype`, `RegExp.prototype`, `Promise.prototype` and the generator prototypes can't be changed at all. The prototypes of `Request`, `Response` and five other Bun classes are read-only | `arr.push(x)`, `map.get(k)` and `Math.floor(x)` compile to the operation, without checking that nobody replaced it | | A program with no call to `eval` or `new Function` is closed. The compiler finds those calls by following the functions as values, so `const run = eval` counts. A call that it couldn't follow, like `globalThis[name](code)`, throws an `EvalError` | Every caller of a function is known. Calls are direct, guards go away, and what nothing reaches isn't compiled | | The executable holds what your program can reach and nothing else. `import()` and `require()` of a computed path find the files that match its constant parts, and the entry points. Other code throws, like a file that doesn't exist. Data files still load | One file, with no `node_modules` beside it. Less code to compile, map and page in | | A module in the executable runs at most once and isn't in `require.cache` | An import is a direct reference to a variable or a function | | Each thread can have 32 GB of objects. `--aot-huge-heap` lifts that | References between objects are 32 bits | You shouldn't have to find any of this in production: | | your program | | --- | --- | | `bun check --aot` | isn't run. Reports everything the build would report. With `--explain=guards` it lists every guard that can fail | | `bun run --aot`, `bun test --aot` | is checked the way the build checks it, then runs from source with built-in objects read-only | | `--aot-log-type-mismatches` | is compiled, and the executable logs every guard that does fail | ## What it doesn't do There's no JIT in the executable. All machine code is in the file, mapped read-only and executable like the `.text` of a C program. On Linux 6.7 and later the executable calls `prctl(PR_SET_MDWE)` once its code is mapped, and from then on the kernel refuses the process any new executable memory. `dlopen()`, `bun:ffi` calls and up to 256 open `JSCallback`s keep working. `cc()` from `bun:ffi` and native addons that generate machine code don't work there. Text that shows up at runtime is interpreted. `eval`, `new Function` and `node:vm` run in JavaScriptCore's interpreter, in the same process, on the same objects as the compiled code. A `RegExp` built from a string at runtime is matched without a JIT. WebAssembly is compiled if it's a `.wasm` file that the program imports, and interpreted otherwise. `--disallow-code-generation-from-strings` turns `eval` off, like Node's flag. The interpreter is never a fallback for code that the build saw. A function that can't be compiled is a build error. Code from disk is opt-in. By default a dynamic `import()` or `require()` only loads code that's in the executable. `--aot-allow-code-from-disk` lets it load a plugin or a config file from disk, which is then interpreted. `--external <package>` does that for one package. The flag has a price, because the compiler can prove less about anything such a file can get hold of: with it, warm Express serves 13% fewer requests per second than in the table above and Koa 16% fewer. The other five don't move. A program with no dynamic import is the same executable either way. There's no profile and no heap snapshot. The compiler never sees your program run. What's hot is estimated from loops and the call graph. None of your program runs at build time, except macros. There's no cross-compilation, and breakpoints don't work. The build makes code for the platform it runs on. In an executable built with `--compile-inspector`, the console, heap snapshots and CPU profiles work. Breakpoints and stepping don't. | Platform | State | | --- | --- | | Linux x64, glibc | Tested | | Linux arm64, Linux musl, macOS arm64, macOS x64 | Tested. Some tests still fail | | Windows x64, Windows arm64 | Tested. Many tests still fail | | FreeBSD, Android | Bun builds there. Not tested yet | ## Full types Publishing a TypeScript library to npm strips the types out of its code. The `.d.ts` keeps only the ones on its exports. So when you compile an app, your code has types and `node_modules` doesn't, and that's where a lot of programs spend their time. Full types is one more file in the package: `index.js` with the types left in. | | code | types | built | | --- | --- | --- | --- | | `src/index.ts` | all of it | all of them | no | | `index.js` | all of it | none | yes | | `index.d.ts` | none | exports only | yes | | **`index.full.ts`** | **all of it** | **all of them** | **yes** | It sits behind a new condition in `exports`. Tools skip conditions they don't ask for, so Node, `tsc`, editors, Vite, webpack, esbuild, `bun run` and `bun build` resolve what they resolve today. ```json "exports": { ".": { "types": "./index.d.ts", "full-types": "./index.full.ts", "default": "./index.js" } } ``` There's one rule: strip the types and you get `index.js`, exactly. The build checks that every time, against the `index.js` that's installed. So the file can't hide code and can't go stale, and if you patch `index.js`, your patch wins. A full-types file that doesn't match or has a type error gets a warning, and the package is compiled without types. A dependency's types never fail your build. `bun compile --library` builds the whole package from your TypeScript: `index.js`, the `.d.ts` files, `index.full.ts` and a `package.json` with the condition in it. `packages/full-types` has full types for packages that don't ship their own, such as react-dom, Express, Fastify and Koa: annotations added, nothing else changed. `docs/bundler/aot-libraries.mdx` has the rest. ## Smaller executables A `--compile` executable is a copy of `bun` with your program attached, so hello world ships a package manager, a test runner, a type checker and several JIT compilers. Nothing in a compiled program can call those, so `--aot` always leaves them out. `--trim` also leaves out the APIs that your program never uses: `bun:sqlite`, `Bun.Image`, `HTMLRewriter`, `Intl` data by locale, WebAssembly, the bundler, the debugger. It removes code, not names. Every global, class and property of a removed feature is still there, so `typeof`, `in` and `instanceof` answer as before. A call into one throws `ERR_FEATURE_REMOVED` and says how to keep it. `--explain=trim` lists each feature, kept or removed, and why. You can't delete code from a linked binary, because dead functions are scattered all through `.text`. So the work happens when `bun` itself is linked. It's linked once, and the reference graph is read out of the object files' relocations, C++, C and Rust alike. `src/trim/features/*.def` name the features and the functions you have to pass through to reach each one. Then it's linked again with the removable sections grouped into page-aligned ranges of their own, and a table of them goes into `bun`. After that, `bun build --compile` just doesn't map those ranges. Nothing that stays moves, so you don't need a linker or a toolchain. ## How it works Under `--aot` nothing is bundled. Each module stays a module, a CommonJS body is a function, and Bun's own `node:` and `bun:` modules are compiled from their TypeScript sources along with your program. 1. `src/aot/load` walks the imports from the entry points with Bun's resolver, loaders, macros and `Bun.build` plugins, and parses each file once. 2. `src/aot/modules` works out the module graph: ES links, which names a CommonJS file exports, the order of evaluation. `src/sema/fold` folds `process.platform` and the like, and the code behind them is dropped. 3. `src/sema`, the type checker behind `bun check`, checks each file. `src/aot/lower` turns its syntax tree and types into an IR in SSA form, file by file, on all threads. 4. `src/aot/link` merges the files into one program. `src/aot/emit` writes it in a format that both repos generate from one schema (`aot/ir/AOTSchema.def` in the engine). 5. JavaScriptCore runs a flow analysis over the whole program, optimizes the IR, generates machine code through B3 and writes an image: code plus read-only data. 6. `src/bundler/aot`, `src/exe_format` and `src/trim` put the image into a copy of `bun`, with symbols for the compiled functions and without what's trimmed. One table says what every built-in takes, returns, reads, writes and may call back. `src/codegen/builtins` makes it from the engine's descriptions and from Bun's own class and module definitions. Bun compiles it, so a change to one of Bun's native classes needs no new build of WebKit. To start reading: `src/aot/compile.rs` and `src/aot/compile/*_stage.rs` are the driver, stage by stage. `test/aot/ir/README.md` and the files under `test/aot/ir/lower/` show a small program and the IR it has to lower to. `src/aot/lower/convert` and `src/aot/lower/layouts` are where types turn into IR. ## What plain Bun pays There's one `bun`, and the compiler is in it. Everyone who installs Bun gets it, whether they compile or not. That's about 30 MB more on disk, and about 23 MB on musl. An executable from plain `bun build --compile` is a copy of `bun`, so each of those grows by as much. This PR also changes code that runs without `--aot`: `node:http`, streams, events and `process.nextTick` in `src/js`, the checker in `src/sema`, the executable format, and shared parts of JavaScriptCore. Review matters most there. ## Testing Most of the added lines are tests. `test/aot/README.md` lists every group of tests, the command that runs it and where a new test goes. | | | | --- | --- | | `test/aot/programs` | Programs, and `test/aot/generate` writes more when the tests run. Each runs three ways that have to agree: regular Bun, compiled, and compiled with its types erased | | `test/aot/programs/nothing-throws-because-of-a-type`, `test/aot/values-against-their-types` | Values that don't fit their types, everywhere a type can appear | | The `variant()` rows of `test/aot/parts.mts` | The programs again with every pass verified, every function hot, every function cold, no inlining, no flow results, 64-bit references, another code layout | | `--aot-validate` | Compiled code checks every fact it relies on at runtime and stops on a miss. The programs run under it too | | `test/aot/collector-stress` | The programs again with a garbage collection at every third slow-path allocation | | `test/aot/other-suites` | test262, JavaScriptCore's tests, Node's tests and Bun's own `test/js`, each file as an executable against regular Bun on the same file | | `test/aot/frameworks` | Builds and serves Next, Nuxt, Astro, SvelteKit, Angular and React Router apps | | `test/aot/servers`, `test/aot/perf` | The servers and idioms above | | `test/aot/analysis-scaling`, `build-scaling`, `startup-scaling` | Build time and startup can't grow faster than the program | | `test/aot/trimmed-against-whole` | Each program built with and without `--trim`. Both have to print and exit the same | | `test/aot/fuzz` | Generators and mutators, against the shipping build, a build with assertions and the validating build | | `test/aot/engine-tests.test.ts` | The engine's own native and IR text tests, linked with Bun's table of built-ins | | `test/cli/aot-docs.test.ts` | Runs every command in the docs that's followed by its output | Of test262's 47,536 files, 33,467 can run as module code, and 31,231 of those pass. Most of the rest try to change a built-in. All of it takes 130 hours on one core. So each run takes a share: one program in ten, picked by the commit's hash, so every commit runs a different tenth. CI's blocking steps keep the pick fixed, so the same tree always runs the same tests. `AOT_SHARE=all` runs everything. Fuzzers still find programs that it compiles wrong. Known failures are `todo`s in `test/aot`, with the reason next to each. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Pins `WEBKIT_VERSION` to oven-sh/WebKit `main` at fcc1ea0973a4 (oven-sh/WebKit#914). oven-sh#45631 was merged on the preview build of that pull request. This is the same source, built from `main`.
…ory, and bound sparse holes in `files()` (oven-sh#47106) ### What does this PR do? I went through the CVEs and security fixes in libarchive's history (and HdrHistogram's) and turned the ones that could apply to `bun_tar`, `bun_archive` and `bun_histogram` into test cases. Two problems came out of it. Both are also in the last libarchive-based canary, so neither is a regression from oven-sh#46638, which itself has only shipped in canary builds. The third change gives `Bun.serve` directory routes the rules they have on Linux with `openat2` everywhere else but on Windows. The fourth stops `fs.realpath` from opening its path on macOS. #### 1. `extract()` followed symlinks that were already in the destination The extractor checks the text of each entry's path (no `..`, not absolute) and the text of each symlink's target. Where a path leads also depends on the symlinks already on disk, and those were followed. This is the class of CVE-2016-4679 in libarchive, and of CVE-2020-11736 and CVE-2020-36314 in file-roller, which uses it. - With `dest/out -> ../outside` in place, an entry `out/new.txt` was written to `outside/new.txt`. The same held for directories, for parents created on the way, for symlink entries, and for a file entry over a symlink (`victim.txt -> ../outside/victim.txt` was truncated and rewritten). - No symlink has to be there beforehand. Three archives extracted to one directory are enough: `a/up -> ..`, then `a/up/esc -> ..`, then `esc/new.txt`. Each target is inside going by the path of its symlink, but `a/up/esc` is really `dest/esc`, so its `..` is the parent of the destination. - On macOS, `mkdirat` and `symlinkat` follow a dangling symlink when the name ends in `/`, which the name of every directory entry does. `dangling/` created a directory outside. The fix is the new `bun_sys::beneath` (`src/sys/beneath.rs`), which the POSIX arms of `Archiver::extract_to_dir`, `create_deferred_symlinks` and `extract_to_disk_filtered` now call in place of `openat`, `mkdirat`, `symlinkat`, `unlinkat` and `make_path`: | | resolved by | | --- | --- | | Linux 5.6+ | `openat2(RESOLVE_BENEATH \| RESOLVE_NO_MAGICLINKS)` | | macOS 26+, FreeBSD | `O_RESOLVE_BENEATH` | | macOS before 26 | `O_NOFOLLOW_ANY`, on a path that `readlinkat` has resolved if it has symlinks | | Linux before 5.6, Android, `openat2` blocked by seccomp | one `openat(O_NOFOLLOW)` per component, on the same | All of them follow a relative symlink that stays inside and refuse every other one. - `mkdirat`, `symlinkat` and `unlinkat` take no flag that constrains resolution. They are given a single name, without a trailing slash, in a parent that was opened under the constraint. - The parent of a new symlink is reached without following any symlink (`RESOLVE_NO_SYMLINKS`, `O_NOFOLLOW_ANY`), because `is_symlink_target_safe` counts the `..` of a target from where the symlink's path says it is. If a symlink is in the way, the parent is resolved as below, the target is checked again from the resolved path, and the symlink is made there, again without following any (`beneath::symlinkat_where_safe`). So under `current -> releases/v1`, the entry `current/bin/x -> ../lib/x` lands in `releases/v1/bin` as it does on `main`, and `a/up/esc -> ..` is refused. This replaces the `st_dev`/`st_ino` comparison in `create_deferred_symlinks`, which only noticed a parent that changed between its two passes. - `beneath::symlinkat` answers `ENOENT` for a name that ends in `/`. No kernel makes `a/l` of `a/l/`, and `is_symlink_target_safe` counts the `..` of such a target from `a/l`, not from `a`. What such an entry leaves behind is unchanged: a directory of that name, or nothing with a glob. - Without the kernel, a path is first opened with no symlink allowed: one `openat` with `O_NOFOLLOW` per component, or `O_NOFOLLOW_ANY` on macOS. That is all a path without symlinks costs. If it meets a symlink that may be followed, the path is resolved and opened again in the same way. Resolving asks `readlinkat` about each component and puts the target of a symlink in its place. A `..`, which only a target can have, drops the name before it and is `EXDEV` when there is none. Nothing depends on the disk staying as it was read: whatever the resolved path names by then, the second open follows no symlink either. It gives up with `ELOOP` after as many symlinks as the kernel follows (40 on Linux, 32 elsewhere). - The walk keeps what a last `/` means to the kernel: with `O_CREAT` it is `EISDIR`, otherwise the name has to be a directory. An empty path is `ENOENT`. - Parents are opened with `O_PATH` / `O_SEARCH`. A symlink entry in a directory that its archive made `0300` used to be dropped, because the parent was opened for reading. - `bun install` pays nothing. It extracts to a directory it has just created, and symlink entries are created after everything else, so there is no symlink to meet. `ExtractOptions::new_destination` says so and keeps the plain syscalls (`Symlinks::Follow`). `TarballStream` is not touched. - With a glob, a file is now opened first and its parents are created only if that fails, as without a glob. That is one `mkdirat` fewer per file. `open` ignores the flags it does not know, so an `O_RESOLVE_BENEATH` that a macOS kernel lacks would silently do nothing. Support is therefore taken from a refusal: the first call asks for `..` of the directory it was given, and only `ENOTCAPABLE` counts. macOS 26 answers that before any permission check (tried with mode `0000` on the directory and on its parent, and on a deleted directory). An error that says nothing about the flag (`EMFILE`, `ENFILE`, `ENOMEM`, `EBADF`, `ENOTDIR`, `EIO`, `ESTALE`, `ETIMEDOUT`) fails that one call and is not remembered. Linux needs no such probe, because a missing `openat2` fails. `openat2_in_root`, which only directory routes called, is now `beneath::openat` with `Symlinks::FollowAsRoot`. Its probe and its "unavailable" cache are in `openat2_if_available`, which all modes share. That also retries `EAGAIN`, which `openat2` answers when anything on the system is renamed while it resolves a `..`, and after four attempts the path is resolved without the kernel. `openat2_in_root` did neither, so a route could answer `404` for `-> sub/../file` now and then. `BUN_FEATURE_FLAG_DISABLE_RESOLVE_BENEATH=1` makes every `openat2` caller behave as on a kernel without it. It is there for seccomp policies that kill the process on `openat2`, which no check from inside the process can find out, and the tests use it to cover the fallbacks. `docs/runtime/archive.mdx` names it. #### 2. `files()` allocated whatever a sparse map declared `MemoryDecoder::read_data` fills the holes of a sparse member with zeros, up to a declared size of 2^60. Holes take no space in the archive, so the usual bound (output is at most the input) does not hold. A 3,584-byte tarball that declares a 3 GiB file made `files()` on that canary return a 3 GiB `File` and grow RSS by 3,076 MiB. `read_data` now counts the holes it has filled per archive and fails with `HolesTooLarge` above 1,024 times the size of the archive as it was given, but not below 64 MiB or above 1 GiB (`holes_limit` in `src/tar/decoder.rs`). 1,024 times is about what gzip makes of the same input, so a hole is worth no more to an attacker than compressed zeros. That covers all of its callers: `files()`, the plucker, `bun pm diff`, `bun publish` and `bun pm pack`. `extract()` is not limited, since it leaves holes unallocated on disk. #### 3. Directory routes followed symlinks out of `dir` without `openat2` `DirectoryRoute::open_beneath` uses `openat2(RESOLVE_IN_ROOT)` on Linux and used a plain `openat` everywhere else, as documented. On macOS, `public/escape -> ../secret.txt` is served with `200`. Directory routes have been in releases since 1.4.0. So did Linux before 5.6 or under a seccomp policy that blocks `openat2`, where `openat2_in_root` fell back to a plain `openat`. What a tree serves should not depend on what the kernel can enforce, so the rules of `RESOLVE_IN_ROOT` now hold on every system but Windows: `dir` is the root directory, an absolute target starts from it, and a `..` at `dir` stays there. Where the kernel does not do that, the resolver above does. For it, these rules are two match arms (`Symlinks::FollowAsRoot`). macOS 26 and FreeBSD still ask the kernel first: what `O_RESOLVE_BENEATH` opens is what `RESOLVE_IN_ROOT` would open, so only a path that it refuses for leaving `dir` is resolved by Bun. Nothing outside of `dir` is opened. On `main`, a request for a symlink to a FIFO outside of `dir` is a `404` that has opened the FIFO. `RESOLVE_BENEATH` alone would stop the same escapes on every system. But it refuses targets that `RESOLVE_IN_ROOT` clamps and serves, such as `-> /inside.txt` and `-> ../../inside.txt` (both checked with the raw syscall on Linux 7.0), and Linux has served those since 1.4.0. #### 4. `fs.realpath` on macOS opened what its path leads to This one is not from the CVE list. It is the same kind of problem as the FIFO in 3: opening something has effects, and needs permissions, that asking where it is should not. `realpath_inner` opens the path, asks the descriptor for its path, and closes it. On Linux the open is `O_PATH`, which gives a descriptor for the place without opening what is there. macOS has no `O_PATH`, so there it was a real `open(O_RDONLY | O_NONBLOCK | O_NOCTTY)`, then `fcntl(F_GETPATH)`, then `close`. So on macOS, for `fs.realpath`, `fs.realpath.native`, their sync forms and `fs.promises.realpath`: | | `main` | this branch | | --- | --- | --- | | a file of mode `000`, or a symlink to it | `EACCES` | the path | | a directory of mode `100` | `EACCES` | the path | | a Unix socket, or a symlink to it | `EOPNOTSUPP` | the path | | any path, with no descriptor left | `EMFILE` | the path, or `ENOENT`, `ENOTDIR` or `ELOOP` | | a FIFO that a writer is blocked opening | the path, and the writer is released | the path, and the writer stays blocked | Node 25.6 answers the first three with the path too. It now asks `getattrlist()` for `ATTR_CMN_FULLPATH` (`bun_sys::get_full_path`): one system call, by path, with no descriptor. `getattrlistbulk()` lists a directory and needs a descriptor for it, so it does not fit here. `getattrlist()` does not always answer what `open` and `F_GETPATH` answer. These are the cases I found, and they still open the path as before: | | `getattrlist()` alone | | --- | --- | | `/.nofollow/…` and `/.resolve/1/…` | repeats the prefix, so `/.nofollow/private/tmp` would be its own real path | | `/dev/fd/3`, `/dev/stdin` | `/dev/fd/3`, and not the file that is open there | | `file/..namedfork/rsrc` | `EPERM` | | an ACL that denies `readattr` and nothing else | `EACCES` | | a path of 1,020 bytes behind `/.resolve/0/` | `ENOSPC`, because the prefix takes room in the kernel's buffer | There may be more, such as a volume that answers `ENOTSUP` or `EINVAL`, which `getattrlist(2)` documents and I have no such volume to try. So the rule is the other way around: `ENOENT`, `ENOTDIR` and `ELOOP` are returned at once, because `open` only repeats them, and every other error falls back to `open`. With that, the only answers that change are errors that become paths, and `EMFILE` for a path that does not exist, which is `ENOENT` now. Node 25.6 answers the same for an absolute path. For a relative one its `process.cwd()` needs a descriptor. It is also faster. In an optimized build, 200,000 calls each on three paths, three rounds: `open`, `F_GETPATH` and `close` took 19 to 52 µs per call, and `getattrlist()` 0.7 to 1.4 µs, which is 16 to 36 times less. `open` is where antivirus and endpoint-security software hooks in, so the ratio depends on the machine. #### Behavior changes - `extract()` rejects (or, with a glob, skips the entry) where it used to write through a symlink that leaves the destination. Directory and symlink entries that need one are skipped. The error says `A symlink in the destination is absolute or leads outside of it`. Its `code` is `"EXDEV"`, and its `path` is where the entry would have been written, relative to the destination, such as `out/new.txt`. The message is a fixed string, so no byte of the archive is in it. - `extract()` never follows a symlink with an absolute target, even if it points inside. Both kernels refuse those. - Directory routes on macOS, on FreeBSD and on Linux without `openat2` resolve symlinks as Linux with `openat2` does. They used to follow them anywhere. A symlink that leads outside of `dir` is no longer served. Neither is one that leads back into `dir` by its real path, such as `-> "$PWD/public/logo.png"` or `-> ../public/logo.png`: those are looked up under `dir` and are a `404`, as they have been on Linux. `-> logo.png`, `-> sub/../logo.png` and `-> /logo.png` are served. So is `-> /etc/hosts` if `dir` has an `etc/hosts`, and `-> ..` is `dir` itself. - On macOS, `fs.realpath` and its variants answer with the path where they used to fail with `EACCES` (what the path leads to cannot be read), `EOPNOTSUPP` (it is a socket) or `EMFILE`. #### Not changed - **Windows** is the one system where the rules still differ. A directory route follows symlinks and junctions wherever they lead. Extraction never creates symlinks there, so an archive cannot plant one, but nothing checks for junctions and symlinks that are already in the destination. None of this was run on Windows, so its arms are as they were. `OBJ_DONT_REPARSE` may be the equivalent to look at. - Every other extraction failure still reaches JavaScript as `ReadError`. That includes a symlink that cannot be resolved at all: a loop, too long a chain, or a target with a missing directory in it. The recorded archives of `archive-golden` expect that message. - `extract()` without a glob still counts the entries it skips. - `fs.realpath` on FreeBSD still opens its path with `O_RDONLY`. `O_PATH`, or libc's `realpath`, may do there. I could only compile that arm, so it is as it was. - `fs.realpath("/dev/stdin")` is still `EBADF` on macOS when stdin is a pipe. Node answers `/dev/fd/0`. - A path that only a compiled program has (`bun build --compile --aot`) still resolves to itself. That rule followed the `open` that macOS no longer does, so there it now follows `getattrlist()`. - Three other places on macOS open a path only to learn where it is: the module resolver for each symlinked entry (`lstat`, `open`, `fstat`, `F_GETPATH`, `close`), `Bun.main`, and `fs.watch` when it starts. `get_full_path` would fit all three. The resolver is a hot path and deserves its own change and benchmark. #### Where resolving without the kernel differs from the kernel None of these leaves the directory. - `fs.protected_symlinks` and the `nosymfollow` mount option apply when the kernel follows a symlink, not when Bun reads one and goes on. - The resolved path is given to the kernel from the destination, so it cannot be longer than `PATH_MAX`. The kernel itself gets deeper than that through symlinks. - On macOS, a symlink with mode `000` and a target that names a resource fork (`f/..namedfork/rsrc`) are followed by the kernel and refused without it. - Some refusals have another error. A symlink that is not followed in the middle of a path is `ELOOP` for `openat2` and `ENOTDIR` without it. A chain of 39 symlinks that ends outside is `ELOOP` and `EXDEV`. To create `./` exclusively is `EEXIST` and `EISDIR`. To create `link/` is `EISDIR` for `openat2`, and without it whatever following the symlink comes to. #### What else was tested and found fine The corpus has 1,282 cases (293 for extraction, 989 for the readers). Each runs through two to four of `extract()`, `extract({ glob })`, `files()` and an entry listing. That is 4,288 jobs in separate processes on macOS, with the destination's surroundings, exit status, time and RSS checked. | | escapes | crashes | hangs | | --- | --- | --- | --- | | `main` (debug + ASAN) | 30 | 2 | 0 | | 1.4.4-canary.1+963452075 (libarchive) | 30 | 0 | 0 | | this branch | 0 | 0 | 0 | | this branch, `BUN_FEATURE_FLAG_DISABLE_RESOLVE_BENEATH=1` | 0 | 0 | 0 | The two crashes are ASAN aborting on a 1 TiB allocation, which is problem 2. The six cases with a 1 TiB sparse file were left out of the run on the canary. - **Readers:** every typeflag, size, time and mode fields at and past their limits in octal and base-256, malformed pax records and out-of-range pax numbers, `L`, `K`, `x`, `g` and `V` headers that are oversized, negative, repeated or cut short, old GNU, pax 0.0, 0.1 and 1.0 and `SUN.holesdata` sparse maps, ACL text, `._` entries, names that are not UTF-8, members whose data is missing, and layered, cut short and damaged gzip. No crash, hang or unbounded allocation other than problem 2. - **Writer** (CVE-2025-5917 and the fixes around it): 3,969 names around the ustar limits of 100, 155 and 255 bytes, with the slash at each position that matters. Bun and `bsdtar` both read all of them back unchanged. - **Histogram:** 2,137 cases from the four HdrHistogram CVEs and the fix history of HdrHistogram_c (bucket configurations, counts that overflow through `add`, top-bucket values, `percentile` inputs, wrong receivers, `structuredClone`). No crash, negative value or percentile out of order. The two decoding CVEs do not apply, because Bun has no decoder. - The records about formats Bun does not read (rar, 7zip, zip, iso9660, cab, lha, mtree, xar, warc, cpio) do not apply. ### How did you verify your code works? New tests in `test/js/bun/archive.test.ts`. The symlink ones run without a glob and with one, by default and with the kernel feature turned off: - `path safety > symlinks in the destination > nothing is written outside of it`: 18 cases - `… > one that stays inside`: 8 cases, among them 30 and 41 symlinks in a row, one to itself and one with `..` in its target - `… > a symlink entry below one that stays inside`: 4 cases, three that are made where they really are and one whose target leads outside from there - `… > directories that no entry names are 0755` (the class of CVE-2023-30571) - `… > in a directory that cannot be listed` - `… > a symlink whose name ends in a slash is not made` - `… > one that goes back and forth 400 times` - `sparse members > more holes than files() reads` In `test/js/bun/archive-golden/archive-golden.test.ts`: - `extract() when the kernel is not asked to resolve paths` runs the recorded archives again with the flag set, so the fallbacks are held to the same results as the kernels. It is a control too: the flag means nothing to the system Bun. In `test/js/bun/http/serve-directory-routes.test.ts`: - `symlinks … > only lead to what is inside the root` takes the place of `rejects symlinks that leave the root`, which was Linux only. It runs everywhere but on Windows, by default and with the flag set, and expects the same 16 answers on every system: symlinks that stay inside, absolute targets at the top and in a subdirectory, a `..` that is the root, real paths that lead back into the root and are not found under it, six ways to leave (one to a sibling whose name starts with the name of the root), and a loop. With `USE_SYSTEM_BUN=1` on macOS both fail. In `test/js/node/fs/fs.test.ts`: - `realpath does not open what its path leads to`: a file of mode `000`, a directory of mode `100`, a Unix socket, a symlink to each and a file in that directory, for all five variants. It first checks that `open` fails on them. It fails with `USE_SYSTEM_BUN=1` on macOS. - `realpath needs no file descriptor`: with none left, a file resolves, and a missing path, one below a file and a loop are `ENOENT`, `ENOTDIR` and `ELOOP`. All eight answers are `EMFILE` with `USE_SYSTEM_BUN=1`. - `realpath leads from /dev/fd to the open file` and `realpath resolves what macOS only resolves for open()` are controls: they pass on `main`, and hold the cases that still open to what `main` answers. The rows with `/.nofollow/` and `/.resolve/` only run where the kernel has those prefixes, which is from macOS 15.4. CI runs macOS 26 and macOS 14, so both the `O_RESOLVE_BENEATH` path and the path of an older kernel run there. The tests in `archive.test.ts` are 34. With `USE_SYSTEM_BUN=1` (1.4.4-canary.1) on macOS, 16 fail and 18 pass. The 18 are controls for the new code rather than for the old bugs: `0755` (4), `ends in a slash` (4), `goes back and forth` (4), `cannot be listed` with a glob (2), `a symlink entry below one` with a glob (2), and `extract()` still writing more than 1 GiB of holes (2). On this branch all 170 tests of the file pass. By hand, with archives made by the system `tar` and the same script on `main` and on this branch: - Each of the escapes above happens on `main` and not here. A normal archive extracts identically, and `bun install ./pkg.tgz` produces the same tree. - While another process keeps swapping `dest/d` between a directory and a symlink to `../outside`, 80 extractions of 200 files under `d/` left 201 entries outside on `main`, and none here, with or without the flag. - 20 extractions over symlinks that are already there: five kinds that stay inside (to a directory, a chain of three, `..` in the target, a file over a symlink, `node_modules/x -> ../store/x`) are followed, and five that do not (absolute, leaving, leaving and coming back, going down first, to itself) are refused. The output with the flag set is the same. - `files()` on a 3,072-byte tarball of a real sparse file reads one of 32 MiB, and rejects one of 65 MiB and one of 3 GiB with no growth in RSS. The canary returns the 3 GiB and grows by 3,075 MiB. - 34 requests to a directory route on macOS 26, on `main` and on this branch. 19 change. 7 reached something outside of `dir` (`SECRET`, `/etc/hosts`) and are a `404` now. 11 led back into `dir` by its real path and are a `404` now, as on Linux. 1, `sub/alias -> /inside.txt`, was a `404` and is served now, as on Linux. The other 15 are unchanged: plain files, redirects, index files, relative symlinks, a `Range` request through a symlink, a loop and dangling symlinks. With the flag set, all 34 answers are the same. - A writer that is blocked opening a FIFO outside of `dir` is released by a request for a symlink to it on `main`, which answers `404`. Here it stays blocked, with or without the flag. - With the fd table full at the first request to a route, so that the probe gets `EMFILE`, that request is a `404`. Once descriptors are free again, the symlinks that leave are still a `404` and the one that stays inside is a `200`. For `fs.realpath`, on macOS 26.7: - I linked `bun_sys` into a small program that resolves a path both ways and ran it over 607,490 real paths, 66,305 of them symlinks: `/usr`, `/private/etc`, `/opt/homebrew`, the system frameworks, `/Applications`, `/System/Volumes/Data`, `/tmp`, `/var`, `/etc` and this repository. 605,295 get the same path and 2,193 the same error. 2 were `EACCES` and get a path now. None gets a different path, and none that had a path gets an error. - With each of the five clauses of `get_full_path` removed in turn (three prefixes, the fallback on an error, the errors returned at once), the same program reports a difference for the cases that clause is there for, and for no other. Those cases are the rows of the tests above, and of the existing `realpath reports ENAMETOOLONG…`. - 22 paths with `realpathSync` and `realpathSync.native`, on 1.4.4 and on this branch: 10 of the 44 answers change, all from an error to the path that Node 25.6 gives. Letter case, firmlinks (`/System/Volumes/Data/Users` is `/Users`), `ENOENT`, `ELOOP` and `ENOTDIR` stay as they were. - `/dev/stdin` and `/dev/fd/0` answer the same on `main` and here with stdin a file, `/dev/null`, a pipe, and closed. - A writer that is blocked opening a FIFO is released by `realpathSync` on `main`, and stays blocked here. With no descriptor left, `main` answers `EMFILE` and this branch the path. A path of 862 bytes resolves and one of 1,063 bytes is `ENAMETOOLONG`, on both. - `fs.test.ts` (622 pass), Node's five `test-fs-realpath*.js`, `promises.test.js` (30) and `fs-mkdir.test.ts` (21) pass. The Linux and FreeBSD arms are not compiled on macOS, so: - `bun_sys`, `bun_tar` and `bun_archive` build for `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-musl`, `aarch64-linux-android`, `x86_64-unknown-freebsd` and `x86_64-pc-windows-msvc`, and `bun_runtime` for `x86_64-unknown-linux-gnu` and `x86_64-unknown-freebsd`. - I linked `bun_sys::beneath` into a static Linux binary and ran 123 calls of it in a container on Linux 7.0, as a non-root user, three times: with `openat2`, with the flag set, and under a seccomp profile that answers `openat2` with `ENOSYS`. Nothing was created outside in any of them. - A fuzzer in the same three situations takes the kernel for the reference. It builds 6,000 forests of directories and symlinks, some with chains around the limit of 40 and targets close to `PATH_MAX`, and opens 60 paths in each to read, search, create or create exclusively: 192,029 with `FollowInside`, 96,025 with `FollowAsRoot` and 71,946 with `Refuse`. Whenever either side succeeds, so does the other, and they have opened the same file: 31,817 times, 1,806 of them something other than what the path says literally, so through a symlink. 15,902 are refused for leaving. Nothing outside was opened or changed. About 1,300 of the 288,054 that may follow symlinks are refused with another error than the kernel's, as listed above. The number moves by a few from run to run, because the kernel's answer for a long chain that ends outside does. - The number of system calls is linear. For one open through `-> x/../x/../…/real`, `strace` counts `2 × pairs + depth + 3` calls of `readlinkat` and `2 × depth + 4` of `openat`: 327, 527, 927 and 1,727 in all for 100, 200, 400 and 800 pairs 40 directories down, and 1,637, 1,667, 1,727 and 1,847 for 800 pairs at 10, 20, 40 and 80. Each `readlinkat` is given a path from the destination, so the kernel's own work per call grows with the depth, as it does for `realpath(3)`. - The tree and the 16 requests of the new route test, opened as the route opens them on Linux, get the answers the test expects, and the same ones in all three situations. macOS gives those answers too. - I linked `bun_archive` the same way and ran `Archiver::extract_to_disk`, which is `extract()` without a glob, on the archives of `archive-golden` in the same three situations. 1,098 of the 1,253 can be compared, and all 1,098 leave what was recorded. The other 155 reach zlib or another library that the binary does not link. Earlier states of this branch differed in 24 of them by default and in another 18 without `openat2`, all from names that end in `/`. CI caught the first kind, in `archive-golden`. - Linked the same way, `extract_to_disk` ran on seven sets of archives with symlink entries below symlinks: the four cases of the new test, `current -> releases/v1`, an entry through a symlink that leaves, and the three archives of `a/up/esc`. In four, the entry stays inside and is made where it really is. In the other three, the entry that would lead outside is not made. Nothing is written outside, and the output is the same in all three situations. - The FreeBSD arm has only been compiled. The macOS constants and semantics (`O_RESOLVE_BENEATH` is `0x1000` and fails with `ENOTCAPABLE`, 107, neither of which the pinned `libc` has) come from the macOS 26 SDK headers and from C probes on macOS 26.7. The macOS 15.4 SDK has the flag too, documented differently and without `ENOTCAPABLE`. That kernel is untested, so only the macOS 26 behavior counts as support and 15.4 takes the `O_NOFOLLOW_ANY` path. Also run on macOS, before the `fs.realpath` change: the source lints (186 pass), `symlink-path-traversal` (93), `bun-pm-cache-pack` (206), `bun-install-streaming-extract` (134), `bun-install-tarball-integrity` (165), `archive-golden` (150), `serve-directory-routes` (31, its 3 Linux tests skipped), `bun-pack` (362) and `bun-create` (131 of 140). The nine failures are the same nine on `main` on the same machine: the LeakSanitizer test times out, and the others run `git commit`, which could not sign on this machine at the time. `perf_hooks` (24) and `bun-install-git-deps` (196 of 200, the same four as on `main`) were run on the first commit only.
…ven-sh#47235) ### What does this PR do? `CrashReason::Panic` held a `&'static [u8]`, but three of its callers only have a borrowed message: `panic_impl(msg: &[u8])`, the Rust panic hook (a stack buffer) and `Bun__crashHandler(ptr, len)`. Each erased the lifetime with `detach_lifetime` in an `unsafe` block. This gives the enum the lifetime it was missing: `CrashReason<'a>` with `Panic(&'a [u8])`. `crash_handler` takes `CrashReason<'_>`, and `TraceString<'a>` stores `CrashReason<'a>`. Nothing keeps a `CrashReason` past the call that made it, so the borrow checker accepts all three callers as they are. The fourth `detach_lifetime` in this file, in `CrashHandler__beginDlOpenAction`, was redundant. `ffi::cstr` turns the raw pointer into a `&CStr` whose lifetime its caller chooses, so the lifetime that the C++ caller's contract allows is now chosen there, in the `unsafe` block that was needed anyway. **4 fewer `unsafe` blocks**, no new code, no cost. The crate no longer uses `detach_lifetime`. `ZigError` keeps `&'static [u8]`: every caller passes a literal. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/cli/run/run-crash-handler.test.ts` | 30 | 1 | yes | | `test/cli/run/crash-report-command-char.test.ts` | 3 | 0 | yes | | `test/js/node/process/dlopen-duplicate-load.test.ts` | 17 | 0 | yes | | `test/regression/issue/15753.test.ts` | 6 | 0 | yes | Total: 56 pass, 1 fail. The one failure is `crash trace starts at the crash site, not inside the crash handler`. It fails the same way on `main`: it looks for `js_panic` in the symbolized trace, and the frame is printed as `bun_runtime::api::crash_handler_jsc::panic`. Of the four call sites, these tests run `panic_impl` and `CrashHandler__beginDlOpenAction`. The Rust panic hook and `Bun__crashHandler` are covered by the type check only. I also triggered a panic, a root error, out-of-memory and a segfault through `bun:internal-for-testing`, and called `process.dlopen` on a missing file and on a file that is not a library, on this build and on `main`. With addresses and timings masked, the first lines of the output and the exit code are the same on both builds in all six cases. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…#47238) ### What does this PR do? The shell reference-counts `IOReader`, `IOWriter`, `PipeReader` and `BuiltinBlob` with `std::sync::Arc`, although none of them leaves its thread. To get that past clippy's `arc_with_non_send_sync`, which is `deny` here, it carried: - `unsafe impl Send` and `unsafe impl Sync` for `IOReader` and `IOWriter`, each commented "shell is single-threaded; `Arc` is used purely for refcounting"; - `#[allow(clippy::arc_with_non_send_sync)]` on `PipeReader`; - a module of trait tricks asserting at compile time that `Arc<PipeReader>` is *not* `Send` or `Sync`; - a compile-time assertion that `BuiltinBlob` *is* `Send + Sync`. This uses `std::rc::Rc` (and `rc::Weak`), which says the same thing in the type. All of the above is deleted. **4 fewer `unsafe impl`s.** Reference counts are plain increments instead of atomic ones. The rest of the diff is the rename (`Arc` to `Rc` in code and comments, `arc_as_mut_ptr` to `rc_as_mut_ptr`) and the two lists that track such impls: the four entries leave `vm-thread-door.inventory.json`, and the two for these files leave `mordant-baseline.toml`. **Does anything carry one to another thread?** The compiler now rejects that wherever `Send` / `Sync` are derived. It cannot see through raw pointers or hand-written `Send` / `Sync` impls, so I read those: - **Thread-pool tasks.** None of the 12 structs that run on the pool has a field of one of these types, or of a type that contains one: `ShellCpTask`, `ShellLsTask`, `ShellMkdirTask`, `ShellMvCheckTargetTask`, `ShellMvBatchedTask`, `ShellRmTask`, `DirTask`, `ShellTouchTask`, `CondExprStatInner`, `ShellGlobTask`, and `cp`'s `NewAsyncCpTask<true>` and `CpSingleTask<true>` in `node_fs.rs`. They hold ids, byte buffers, fds, atomics and raw pointers. - **The interpreter pointer.** The interpreter does own these `Rc`s, and nine of those structs embed a `ShellTask`, which holds a `*mut Interpreter`. None of the functions that run on the pool (`run_from_thread_pool`, `run_from_thread_pool_impl`, `work_pool_callback`, `cp_on_copy`) dereferences it. - **`JsCell`** is `Send + Sync` whatever it holds, so the compiler would not object to moving a struct that keeps an `Rc` in one. In the shell, the only ones that do are fields of `Interpreter`, covered above. - **Callbacks.** `IOReader`, `IOWriter` and `PipeReader` give a raw pointer to themselves to their `bun_io` reader or writer, and a subprocess gives one to its `Process` for the exit callback. The event loop that owns the shell dispatches those callbacks on its own thread. The GC's size callback returns a cached number and does not walk these objects. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build: All 44 test files under `test/js/bun/shell/` (including `commands/`): **1053 pass, 3 fail**. The three failures are timeouts: `leak.test.ts` `memleak_Blob_something` and `memleak_Blob_nothing` (100 s limit) and `shell-load.test.ts` `immediate exit` (90 s limit). Run alone, they time out the same way on a debug + ASAN build of `main` on this machine, which is too slow for those limits. Because that leaves the `Blob` leak tests unfinished here, I ran a smaller loop on both builds: 480 shell commands with a 128 KB `Blob` on stdin, after a warm-up, with ASAN's quarantine turned down to 1 MB. RSS changed by -0.4 MB on this build and +2.4 MB on `main`. Leaking the blob once per command would be about 60 MB. I also ran one script on both builds that covers pipelines between builtins and processes, redirects to and from files, a `Response` on stdin, a `Buffer` on stdout, command substitution, `ls -R`, `cp`, `mv`, `rm`, a 20,000-line pipe, 20 concurrent commands, `.quiet()` and `.lines()`. The output is the same on both. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…-sh#47231) ### What does this PR do? `uws::Loop` mirrors the C `us_loop_t` field by field, and its methods took `&mut self`. So `tick(&mut self)` claimed exclusive access to the loop for the whole tick, while C and the callbacks it dispatches write the same fields and reach the loop again through `Loop::get()`. The comment on the `extern "C"` block named that problem as the reason every declaration stayed `*mut Loop`, which put an `unsafe` block in every method. This puts every field of `PosixLoop`, `WindowsLoop` and `InternalLoopData` in a `Cell`. `Cell<T>` has the layout of `T`, so the structs still match C, and the layout assertions are unchanged. A `&Loop` now claims nothing about the bytes behind it, so: - every method takes `&self`; - a C function is declared `safe fn(&Loop)` where a live loop on its own thread is all it asks for (six of them), and the `unsafe` block around each call is gone; - code that kept a raw loop pointer for a whole function (`auto_tick`, `auto_tick_active`, the timer code, the HTTP thread) borrows it once as `&Loop`, instead of dereferencing it in an `unsafe` block per call. Four C functions that take only a loop are still `unsafe` to call, because they ask for more: - `uws_loop_date_header_timer_update` and `uws_loop_flush_pending_writes` read uWS's `LoopData` behind the loop. Every loop has one except spawnSync's, which `Loop::create` makes without it. `SpawnSyncEventLoop::uws_loop()` is private, so that loop does not leave its file. - `us_internal_free_closed_sockets` must not run while the loop is dispatching events. - `us_wakeup_loop` is the one that other threads call, so it takes a raw pointer. **29 fewer `unsafe` blocks.** No new `unsafe fn`, `unsafe impl` or helper. `Cell::get` / `set` are plain loads and stores, so the generated code is no worse. What else changes, each needed or made possible by the above: - Fields that nothing outside the crate uses are private. Other threads write `pending_wakeups` and `dns_ready_head`, so Rust must not read them. - `fd`, `head`, `quic_next_tick_us` and (Windows) `uv_loop` are private with read-only getters. C follows these, and `uv()`'s SAFETY comment relies on `uv_loop` never being reassigned, which a `pub` `Cell` would let safe code do. - libuv's `Loop::active_handles` is a `Cell`, and `inc` / `dec` / `add_active` / `sub_active` take `&self`. `WindowsLoop::uv_mut`, which turned the raw pointer into `&mut uv::Loop`, is deleted. - `SpawnSyncEventLoop::uws_loop_mut` is deleted. It now ticks through `uws_loop()`, whose borrow is bound to `&self`. - `HTTPThread::uws_loop_mut`, which returned a `&mut Loop` of any lifetime and held it across `tick()`, is replaced by `uws_loop_ref(&self) -> &Loop`. - `WindowsWaker::wait` calls `run()`. It called the C function directly only to avoid a `&mut WindowsLoop`, so `us_loop_run` is no longer re-exported. - `posix_event_loop.rs`'s private `loop_add_active` / `loop_sub_active` are deleted in favour of the `Loop` methods their comment was waiting for. - `DateHeaderTimer` calls `should_enable_date_header_timer()` instead of reading the field. Not changed: `Loop::get()` still returns `*mut Loop`, and a caller that holds a raw pointer still turns it into a reference in `unsafe`. The other accessors that return `&mut Loop` (`VirtualMachine::uws_loop_mut` and `platform_loop_opt`, `bun_io`'s `platform_event_loop`, `loop_mut` and `uws_loop_mut`) are untouched. ### How did you verify your code works? Release builds behave the same, so there is no new test. In debug builds, activating a `FilePoll` now goes through `Loop::add_active` / `sub_active`, which log under the `Loop` scope. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`. Each file has the same passes and the same failing tests, by name, on both builds. | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/spawn/spawnSync.test.ts` | 18 | 0 | yes | | `test/js/bun/spawn/spawn.test.ts` | 182 | 0 | yes | | `test/js/web/timers/setTimeout.test.js` | 35 | 3 | yes | | `test/js/web/timers/setInterval.test.js` | 7 | 1 | yes | | `test/js/web/timers/setImmediate.test.js` | 3 | 0 | yes | | `test/js/bun/http/serve.test.ts` | 334 | 0 | yes | | `test/js/bun/http/serve-http3.test.ts` | 84 | 0 | yes | | `test/js/bun/http/bun-serve-date.test.ts` | 1 | 0 | yes | | `test/js/web/fetch/fetch.test.ts` | 397 | 1 | yes | | `test/js/bun/net/socket.test.ts` | 194 | 1 | yes | | `test/js/bun/net/tcp-server.test.ts` | 11 | 0 | yes | | `test/js/web/websocket/websocket.test.js` | 97 | 2 | yes | | `test/js/node/dns/node-dns.test.js` | 144 | 28 | yes | | `test/js/node/quic/quic-endpoint.test.ts` | 9 | 0 | yes | | `test/js/node/quic/quic-stream.test.ts` | 13 | 0 | yes | | `test/js/bun/udp/udp_socket.test.ts` | 234 | 0 | yes | | `test/js/node/worker_threads/worker_threads.test.ts` | 161 | 1 | yes | | `test/cli/test/isolation.test.ts` | 50 | 0 | yes | Total: 1974 pass, 37 fail. The 37 failures are the same 37 tests on `main` on this machine, so none comes from this change. I also ran one script on both builds that serves HTTP and a WebSocket, fetches, spawns (sync and async), uses an unref'd interval, a timeout, an immediate and a Worker, then stops the server. Both print the same output and exit 0 on their own, which they would not if the loop's ref counts were off. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…sh#47247) ### What does this PR do? Removes seven `unsafe { detach_lifetime(..) }` blocks. In each case the data already lives as long as the code needs. The signature, field or local just didn't say so, and once it does the borrow checker accepts the code as it is. | Where | Before | After | |---|---|---| | `P::value_for_define` | took `&DefineData` and erased `original_name()` to `'a` | takes `&'a DefineData`, which is what all three callers have (`p.define: &'a Define`) | | `ColonListValue::resolve_value` (`--define`, `--loader`) | took `&[u8]` and erased it to `'static` | takes `&'static [u8]`: the caller's input is already `'static` (argv) | | `edit_trusted_dependencies` | borrowed from a `Query` that died at the end of the `if let`, and erased the borrow | keeps the `Query` in a local for as long as the slice is used, and reuses it where the function looked the same key up a second time | | `DirEntryResolveQueueItem::safe_path` | `RawSlice<u8>`, erased back to `'static` when read | `&'static [u8]`, the type of the `DirEntry::dir` it is set from | | `PackageManager::global_link_dir_path` | interned the path, copied the result into a `Box<[u8]>`, then erased a borrow of the box | keeps the interned `&'static [u8]` | | `AsyncModule::resume_loading_module` (2 blocks) | took `parse_result` out of `self`, put it back, took it out again, and erased two borrows of `self` to get past the write | takes it out once | **7 fewer `unsafe` blocks**, +39/−80. No new `unsafe` and no new helper. Nothing does more work at run time, and three places do less: `global_link_dir_path` saves an allocation and a copy, `edit_trusted_dependencies` a property lookup, and `resume_loading_module` a placeholder `ParseResult`. `unsafe_path`, the other field of `DirEntryResolveQueueItem`, stays a `RawSlice`: it points into a thread-local buffer. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. The change has no platform-specific code. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/transpiler/transpiler.test.js` | 258 | 0 | yes | | `test/cli/install/bun-prune.test.ts` | 177 | 0 | yes | | `test/cli/install/bun-pm.test.ts` | 154 | 0 | yes | | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/cli/install/bun-link.test.ts` | 57 | 0 | yes | | `test/bundler/cli.test.ts` | 44 | 0 | yes | | `test/cli/install/bun-install-link-path.test.ts` | 49 | 0 | yes | | `test/bundler/bundler_loader.test.ts` | 62 | 0 | yes | | `test/cli/install/isolated-relink.test.ts` | 6 | 0 | yes | | `test/js/bun/resolve/import-meta.test.js` | 34 | 0 | yes | | `test/js/bun/resolve/resolve-ts.test.ts` | 27 | 0 | yes | | `test/js/bun/resolve/resolver-permission-denied-ancestor.test.ts` | 2 | 0 | yes | | **Total** | **1132** | **1** | | The one failure is `loads reflect-metadata before tsyringe`. It needs the `tsyringe` package, which is not installed on this machine, so it fails the same way on `main`. I also ran these on both builds and compared the output, which is the same in all of them: - **Defines and loaders:** `--define` with identifier, dotted, string and number values, at run time, in `bun build` and through `Bun.Transpiler`, including assignment and `delete` targets. `--loader` with two mappings. The error messages for a malformed `--define` and `--loader`. - **Resolver:** `Bun.resolveSync` from six directories of a nested project, a `FileSystemRouter` over part of it, an entry point deep in the tree, and `bun build` from a nested directory. A breakpoint confirms that this reaches the branch that sets a non-empty `safe_path`. - **`bun link`:** register, use, `--save`, reinstall, the isolated linker, a package that was never linked, and `bun unlink`. - **`trustedDependencies`:** `bun add --trust` and `bun pm trust` from 14 starting states: no list, an empty list, a list of other names, a name that is already listed, several names at once, `--all`, an unknown name, and a list that is a string, an object or holds non-strings, which are all rejected. **Not covered by any run:** `resume_loading_module`. Nothing reaches it today. It only runs for a module with `pending_imports`, and nothing fills that list, so auto-install at run time goes through the synchronous path (I checked with `BUN_DEBUG_AsyncModule=1`). That change is checked by the compiler only. `self.parse_result` is left as `ParseResult::empty` on both the success and the early-error path, as before. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ntext (oven-sh#47251) ### What does this PR do? While it prints a file, the bundler's printer asks the linker about other files through `RequireOrImportMetaCallback`, a type-erased context pointer plus a function pointer. Two things about that type were wrong, and `print_code_for_file_in_chunk_js` paid for both: - **It demanded `&mut`.** `init` took `&mut T` and the trait method took `&mut self`, but the linker's implementation only reads four graph columns. - **It had no lifetime.** `init` and `call` are safe functions, yet nothing tied the stored pointer to the borrow it came from. Because of the `&mut self`, every other borrow of `self` that goes into the same `js_printer::Options` (`ts_enums`, `import_member_bindings`, the line offset table, `mangled_props`) had its lifetime erased with `detach_lifetime_ref`, and `parse_graph` was dereferenced raw instead of through its accessor. Now the callback is `RequireOrImportMetaCallback<'a>` and holds a `&'a T`, using the `'a` that `Options<'a>` already has, and the trait method takes `&self`. All the borrows are shared, so the borrow checker accepts them as written and the 28-line workaround goes away. With that in place: - The two functions that build a callback, `print_code_for_file_in_chunk_js` and `generate_entry_point_tail_js`, take `&LinkerContext`, which is all they need. - `Options` holds `Option<RequireOrImportMetaCallback<'a>>`, like the `Option<SourceMapHandler<'a>>` next to it. Before, "no callback" was a null context plus a `noop` function that was never called, tested with `is_none()` and then `unwrap()`. - `LinkerContext::require_or_import_meta_for_source` had one caller, the trait method that forwarded to it. Its body is now that method. **5 fewer `unsafe` blocks**, +59/−113, part of which is that moved body. The one `unsafe` block that remains is the cast back from the erased pointer, and its SAFETY comment no longer has to rely on what callers promise. There is no cost. `NonNull` gives the `Option` a niche, so the field is still two words, and a call is still one null test and one indirect call through the same kind of function pointer. I kept the hand-rolled function pointer. `Option<&'a dyn RequireOrImportMetaSource>` would remove the last block as well, but each call would then load the function pointer from a vtable first. Two other lifetime workarounds nearby are left alone: the `Renamer` transmute in the same function and the `import_records` erasure in `postProcessJSChunk.rs`. Both come from `Renamer<'r, 'src>` being invariant in `'src`, which is a separate problem. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes on Linux. The change has no platform-specific code. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/esbuild/importstar.test.ts` | 99 | 0 | yes | | `test/bundler/esbuild/ts.test.ts` | 59 | 0 | yes | | `test/bundler/bundler_cjs2esm.test.ts` | 113 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/bundler_merged_enum.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_minify.test.ts` | 67 | 0 | yes | | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | **Total** | **558** | **35** | | All the failures are in the two `test/bake/dev` files. Those tests drive a browser-like client that needs the `happy-dom` package, which is not installed on this machine, so they fail the same way on `main`. The tests in those files that only need the server pass. I also ran 14 `bun build` commands on both builds over a small project, and then ran five of the bundles. The project has `require()` of ESM and of CommonJS, `import()` of a local file, `import *`, TypeScript enums used across files, a JSON import, a CSS module and top-level await. The commands cover `--format` esm, cjs and iife, `--target` bun, node and browser, `--minify`, `--minify-syntax --minify-whitespace` and `--minify-identifiers`, inline and external source maps, `--splitting` with two entry points, and `--format=internal_bake_dev`. One command, CommonJS output of a module with top-level await, is rejected with the same error. All 4,407 lines of output, which include the bundles and the source maps, are the same on both builds. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…en-sh#47253) ### What does this PR do? The EVP-backed hashers in `bun_sha_hmac` (`SHA1`, `SHA256`, `SHA512`, ...) have a one-shot `hash(bytes, out, engine)`. It was an `unsafe fn` for one reason: `engine` is a `*mut ENGINE`. So all 15 call sites were `unsafe` blocks, 14 of them only to pass a null pointer, several under a paragraph explaining why null is fine. BoringSSL never reads that argument. `EVP_Digest` hands it to `EVP_DigestInit_ex`, whose body does not mention it (`vendor/boringssl/crypto/fipsmodule/digest/digest.cc.inc`). The parameter exists for OpenSSL source compatibility. This drops the parameter. `hash(bytes, out)` passes null itself and is a safe function, with the same signature as the non-EVP hashers in the same file. The one caller that passed a real engine is `Bun.SHA256.hash()` and its seven siblings in `CryptoHasher.rs`. They no longer fetch the VM's engine first, which on the first call also meant creating it. `Bun.CryptoHasher` goes through `EVP.rs` and is not touched: dropping the engine there reaches into the VM's `RareData` and three functions exported to C++, so it is a change of its own. Also gone, because nothing is left for them to do: - `s3_signing`'s `boring_engine()` helper, which returned null, and the `ENGINE` re-export that only it used. That was the last use of `sha::ffi` from another crate, so the module is `pub(crate)` now. - `exe_format`'s `sha256_hash`, which wrapped the `unsafe` call. Its six callers call `SHA256::hash` directly. One of them, the Mach-O signer, hashed each page into a temporary and copied it into the signature. It now hashes straight into the slot. **15 fewer `unsafe` blocks and 1 fewer `unsafe fn`** (a macro stamps it out once per hasher), +34/−102. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes on Linux. The change has no platform-specific code. - `cargo test -p bun_exe_format` passes (237 tests). It covers the hashing of Mach-O code signatures and of build ids. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/util/bun-cryptohasher.test.ts` | 109 | 0 | yes | | `test/cli/install/bun-install-tarball-integrity.test.ts` | 165 | 0 | yes | | `test/cli/install/bun-pm.test.ts` | 154 | 0 | yes | | `test/js/bun/s3/s3-storage-class.test.ts` | 9 | 0 | yes | | `test/js/bun/s3/s3-requester-pays.test.ts` | 10 | 0 | yes | | `test/js/bun/s3/s3-list-objects.test.ts` | 40 | 0 | yes | | `test/js/sql/postgres-auth.test.ts` | 101 | 0 | yes | | `test/js/sql/sql-mysql-auth-short-nonce.test.ts` | 10 | 0 | yes | | `test/js/sql/sql-mysql-duplicate-auth-switch.test.ts` | 6 | 0 | yes | | `test/js/sql/postgres-duplicate-auth-request.test.ts` | 17 | 0 | yes | | `test/js/sql/wire-frames.test.ts` | 10 | 0 | yes | | `test/js/sql/sql-mysql.auth.test.ts` | 37 | 0 | yes | Total: 668 pass, 0 fail. I also ran two scripts on both builds, with the same output: - Each of the eight `Bun.SHA*` / `Bun.MD*` classes on five inputs (empty, 3 bytes, 64 and 65 bytes, 1 MB): `hash()` to hex and into a caller's buffer, compared with the incremental digest. Also `Bun.CryptoHasher`, and `node:crypto` as a cross-check. - `bun install` of a local tarball, which computes its sha512. `bun pm hash` and `bun pm hash-string`. A `--frozen-lockfile` reinstall, which verifies the integrity. The same reinstall after I corrupted the integrity in `bun.lock`, which fails with `Integrity check failed`. `postgres-auth.test.ts` authenticates with SCRAM-SHA-256 against a real Postgres server, and `sql-mysql.auth.test.ts` with `mysql_native_password` and `caching_sha2_password` against a real MySQL server. Those cover the call sites in `SASL.rs` and `Auth.rs`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…afe (oven-sh#47250) ### What does this PR do? `Entry::kind` and `Entry::symlink` stat a cached directory entry the first time someone asks. Both were `unsafe fn`s because they took `fs: *mut R` and formed `&mut *fs` to call `EntryKindResolver::resolve_kind(&mut self, ..)`. So each of their 22 call sites was an `unsafe` block with a comment saying that the pointer is the global `RealFS`. `resolve_kind` does not need `&mut`. Its only implementation, `RealFS::kind`, reads `self.cwd`, calls `self.need_to_close_files()` (a `&self` method) and makes syscalls. So: - `RealFS::kind` and `EntryKindResolver::resolve_kind` take `&self`. - `Entry::kind` and `Entry::symlink` take `fs: &R` and are safe functions. Their `# Safety` section asked for two things. A valid pointer is what the reference now states. That `resolve_kind` does not re-enter the entry's mutex is about a deadlock, not about memory safety, and is now a doc comment on `resolve_kind`. - Callers pass a shared reference: `Resolver::fs_ref()`, `vm.fs()` or a local `&FileSystem` where they have one, and `FileSystem::get()` where they had only the raw pointer. In the resolver it is spelled `self.rfs_ref()`, a one-line safe accessor for `&self.fs_ref().fs`. **One caller needed more than that.** `bun test`'s file scanner calls `kind` from the per-entry hook of `RealFS::read_directory_with_iterator(&mut self)`, so while the hook runs, that `&mut RealFS` is live further up the stack and no independent `&RealFS` can honestly exist. The hook is now lent the borrow: `readdir`, which never used its `&mut self`, takes `&self` and passes itself to `DirEntryIterator::next`. For that, the hook call moves from `DirEntry::add_entry_with_store` up into `readdir`, its only caller with a real hook. `add_entry_with_store` returns the entry it stored and loses its type parameter, and `IS_VOID` and the forwarding `impl DirEntryIterator for &T` have no use left. The hook gets the entry as `&Entry`. It only calls `&self` methods on it, and the map's key can point into the entry, so a `&mut` there would be wrong. This also removes one source of aliased `&mut`. `Entry` is built for use from several threads: it has a per-entry mutex and a lock-free fast path, and most `kind` calls in the resolver run outside the global resolver mutex. Two threads that stat different entries each made their own `&mut` to the one global `RealFS`. Other code still forms `&mut RealFS` from a raw pointer, for `read_directory` and the entries map. That is not changed here, except for two places next to a `kind` call that formed one only to lock `entries_mutex`, which takes `&self`. With `kind` an ordinary `&self` method, some code around it is no longer needed: - Four locals in the resolver that held `rfs_ptr()` only to feed `kind`, three of them with a paragraph on Stacked Borrows. A fifth, in `load_as_file`, is down to one use and is inlined, without its paragraph. - Four `let entry = lookup.entry();` locals that were used once, under a `SAFETY` comment although the call is safe. - `ResolverLike::fs_impl`, which has no users left. - In `RouteLoader::load`, three separate `unsafe { &*entry_ptr }` reborrows that existed "so `&` reads and the `kind()` call do not overlap". One remains. - `Route::parse` was an `unsafe fn` taking `*mut Entry` because "a `&mut Entry` parameter would alias" its other arguments. All it ever did with the pointer was `&*entry`, so it takes `&Entry` and is safe. Its two remaining internal `unsafe` blocks depend only on lengths that it computes from its own buffer. **30 fewer `unsafe` blocks and 3 fewer `unsafe fn`s**, +91/−251 over 8 files. No new `unsafe`. A reference and a raw pointer are passed the same way, and the hook call is in the same loop as before, so there is no cost. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | `test/js/bun/util/filesystem_router.test.ts` | 51 | 0 | yes | | `test/cli/run/filter-workspace.test.ts` | 131 | 0 | yes | | `test/bake/framework-router.test.ts` | 37 | 0 | yes | | `test/cli/bun.test.ts` | 48 | 0 | yes | | `test/js/bun/resolve/import-meta.test.js` | 34 | 0 | yes | | `test/js/bun/resolve/resolve-ts.test.ts` | 27 | 0 | yes | | `test/js/bun/resolve/resolver-permission-denied-ancestor.test.ts` | 2 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/bundler/esbuild/tsconfig.test.ts` | 21 | 0 | yes | | **Total** | **659** | **3** | | The three failures are the same three on `main`: - `loads reflect-metadata before tsyringe` needs the `tsyringe` package, which is not installed on this machine. - `instances of a class nested too deeply to print are a RangeError` and `the data of events nested too deeply to print are a RangeError` test a stack overflow in the value printer, which this PR does not touch. I also ran these on both builds, in directories that contain plain files, directories, symlinks to files, symlinks to directories and dangling symlinks. The output is the same: - **`bun test` discovery**, which is the path that goes through the hook: ten runs over a tree with nested directories, `node_modules` and dot directories, a directory named like a test file, a FIFO, and symlinks that lead out of the project. No filter, name filters, a path, a filter that matches nothing, `--path-ignore-patterns`, and runs from two subdirectories. Every test file prints its own path, so this compares the set of files found. - **Resolver:** `Bun.resolveSync` on 38 specifiers (a package `main` without extension, `exports`, `dir/index`, a file and a directory with the same name, `.js` that resolves to `.ts`, `tsconfig` paths), with and without `--preserve-symlinks`, then the same program through `bun build` and run again. Also a build with missing imports, an unreadable directory, and a file used as a directory. - **`Bun.FileSystemRouter`:** its routes, nine `match()` calls, and `reload()` after a file is added. - **`bun run --filter`** with workspace globs, five filters. - **`bun getcompletes`** for bins, scripts and files. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…h#47249) ### What does this PR do? `bun_core::ffi::zeroed::<T>()` is the safe way to zero-initialize a value: it needs `T: Zeroable`, and `impl_zeroable!(T)` only compiles if rustc can prove from the layout that all-zero is a valid `T`. 27 places zeroed a value with `unsafe` and a hand-written SAFETY comment, through `zeroed_unchecked()` or `MaybeUninit::zeroed().assume_init()`, for types that need no escape hatch. - **14 sites only needed the safe call**, because the type is already `Zeroable`: `sockaddr_storage` (3), `addrinfo`, `statfs`, `[pollfd; 2]`, `EVP_MD_CTX`, and on Windows `FILETIME` (4), `CONSOLE_SCREEN_BUFFER_INFO`, `BY_HANDLE_FILE_INFORMATION` and the directory iterator's name buffer. That last one is an associated type, which now has the bound `NameData: Zeroable`. - **7 sites needed the proof to be declared:** `libc::sockaddr_un`, `libc::rusage_info_v4`, `uv_utsname_t` (2), BoringSSL's `SHA_CTX` and `SHA256_CTX` (one site, in a macro), lsquic's `NqTransportParams` and `bin::Value`. Each is one `impl_zeroable!` entry that the compiler checks. For `bin::Value`, a union with a `()` member, the check proves little. Its other members are `Zeroable` through impls of their own. - **6 sites read better without zeroing at all:** `timespec { sec: 0, nsec: 0 }` in `Futex.rs`, a three-field literal for `Iccp` (the file already builds one that way in the encoder), `LastResult { e: 0 }` in `NativeBrotli.rs`, `static MUTEX: Mutex = Mutex::new()`, a literal for `uv_stdio_container_t`, and `EventLoopHandle::init(null)` for the placeholder in `WindowsOptions::default()`. That last one zeroed a `repr(Rust)` enum and relied on which variant is discriminant 0. `init(null)` is the sentinel that type documents for this use. The hashers in `sha.rs` zero their inner C context, `Self { hasher: zeroed() }`. The public wrapper types are not `Zeroable`, so `init()` stays the only way to make one. Four more values, three `sockaddr_storage` in `node_cluster_binding.rs` and a `winsize`, sit inside `unsafe` blocks that are still needed for the call next to them. They use the safe call too, and the count does not include them. **27 fewer `unsafe` blocks**, +75/−106 over 19 files. No hand-written `unsafe impl`, no new helper, and the same bytes as before. The 9 `zeroed_unchecked()` calls that remain hold pointers or are generic, which is what the escape hatch is for. Comments, in the files this touches: ten `// SAFETY: all-zero is valid` comments sat above calls that were already the safe `zeroed()` and are removed, and six `// SAFETY:` comments in `node_cluster_binding.rs` that stopped in the middle of a sentence are completed. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. For the eleven Windows-only sites and the one macOS-only site this type check, which includes the layout proofs, is the only verification I could do. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/image/image.test.ts` | 104 | 0 | yes | | `test/js/node/zlib/zlib.test.js` | 473 | 0 | yes | | `test/js/bun/udp/dgram.test.ts` | 62 | 0 | yes | | `test/js/bun/util/bun-cryptohasher.test.ts` | 109 | 0 | yes | | `test/js/bun/spawn/spawnSync.test.ts` | 18 | 0 | yes | | `test/js/node/cluster/worker-set-up-by-the-first-to-ask.test.ts` | 1 | 0 | yes | | `test/cli/install/bun-patch.test.ts` | 232 | 0 | yes | | `test/bundler/cli.test.ts` | 44 | 0 | yes | | `test/js/web/websocket/websocket-accept-header-validation.test.ts` | 2 | 0 | yes | | **Total** | **1045** | **0** | | I also ran a script on both builds. Its output is the same: - **Hashers:** the eight `Bun.SHA*` and `Bun.MD*` classes, one-shot and incremental, over five inputs from empty to 1 MiB. `Bun.CryptoHasher` with eight algorithms, including `copy()`. `node:crypto` hashes and HMACs. A WebSocket handshake, where the client checks `Sec-WebSocket-Accept` with the `SHA_CTX` hasher. - **`bun patch --commit` with `BUN_FEATURE_FLAG_DISABLE_MEMFD=1`.** On Linux the output of `git diff` normally goes to a memfd. With that off it is read through the `[pollfd; 2]` loop, and `strace` confirms the `poll` calls. The patch is then hashed with the `SHA_CTX` hasher. This compares the patch file, `package.json`, and a reinstall that applies the patch. - **`bun build` with `BUN_FEATURE_FLAG_FORCE_IO_POOL=1`**, the only mode that uses the `static MUTEX`: 41 modules, and the bundle is then run. - **`fs.statfsSync`**, also with `bigint` and on a missing path, and a **Brotli** round trip and decode error. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
oven-sh#47273) ### What does this PR do? Five separate changes. Each has its own commit, and a sixth commit holds small follow-ups to 2, 4 and 5. Each change removes hand-written pointer code. Most of it becomes plain borrows that the compiler checks. The rest goes through safe constructors and accessors that already exist. Together: **70 fewer `unsafe` blocks**, +299/−765 over 58 files. There is no new `unsafe`, no safe function became `unsafe`, and nothing is added at run time. | # | Change | `unsafe` blocks | Diff | |---|---|---|---| | 1 | The bundler reads source files through `&mut cache::Fs` | −8 | +35/−97, 6 files | | 2 | `NonNull`s are made from references and boxes without `new_unchecked` | −16 | +56/−148, 17 files | | 3 | Four callback contexts borrow instead of holding raw pointers | −7 | +36/−78, 5 files | | 4 | JS codegen workers borrow the linker, the chunk and the renamer with `&` | −17 | +108/−285, 12 files | | 5 | Readers of the `FileSystem` singleton call `FileSystem::get()` | −22 | +67/−160, 22 files | The commits can be reviewed one at a time. Four files are touched by two commits: `bundler/transpiler.rs`, `resolver/lib.rs` and `resolver/resolver.rs` by 1 and 5, and `jsc/VirtualMachine.rs` by 3 and 5. Commits 1 to 3 come from oven-sh#47261, oven-sh#47265 and oven-sh#47266, which this PR replaces. #### 1. The bundler reads source files through `&mut cache::Fs` `cache::Fs::read_file_with_allocator` took `_fs: &mut FileSystem`. All it does with it is call `need_to_close_files()`, which takes `&self`, and all five callers passed the process-wide singleton. To hold a `&mut` to that singleton next to the `&mut` borrow of the cache, they went through `unsafe`. - **The parameter is gone.** The function asks `FileSystem::get()`, the existing shared accessor. - **The bundler's source readers take what they use.** `get_code_for_parse_task`, `get_code_for_parse_task_without_plugins` and `OnBeforeParsePlugin` carried a `*mut Transpiler` and a `*mut Resolver`. The pointers were raw because the resolver is a field of the transpiler, so a `&mut` to each would alias. They used the transpiler only for `fs` and the resolver only for `caches.fs`, so they now take `&mut cache::Fs`. - **`get_source_code` is plain safe code.** It borrows `transpiler.options.loaders`, then `transpiler.resolver.caches.fs`, then calls `transpiler.reset_store()`, and the borrow checker accepts that as written. - `get_code_for_parse_task_without_plugins` also took a `_loader` that it never read. That is gone too. - `PackageJSON::parse` still needs a `&FileSystem` for paths. It gets it from `FileSystem::get()` instead of `unsafe { &mut *r.fs() }`. #### 2. `NonNull`s from references and boxes without `new_unchecked` 22 calls of `NonNull::new_unchecked(p)` were on a pointer that the code had just taken from a reference, a slice, a `Box`, a `Vec` or a `RefPtr`, with a SAFETY comment saying that such a pointer is never null. The standard library and `bun_core::heap` already have safe constructors for exactly that. The repo calls `NonNull::from` about 150 times and `heap::alloc_nn` / `heap::into_raw_nn` about 30 times. `NonNull::from_ref` and `from_mut` are the same conversion as `NonNull::from`, and unlike it they can be called in a `const fn`, which four of these sites are. | From | Was | Now | |---|---|---| | a reference or slice (`StoreRef::from_static`, `StoreStr::new`, `StoreSlice::new` and `new_mut`, `StringHashMapKey::borrowed`, two in `finish_arena_contents`, `crypto.randomBytes`) | `new_unchecked(s.as_ptr().cast_mut())` | `NonNull::from_ref(s).cast()` / `from_mut` | | a `Box` that is given up (`BSSList`'s overflow block, `StringHashMapKey::owned`) | `new_unchecked(Box::into_raw(b))` | `Box::into_non_null(b)` / `Box::into_non_null_with_allocator(b).0` | | a new heap object (`ConcurrentTask::create`, the spawn wait queue, `readdir`'s result list, the FSEvents loop, `IPCInstance::new`, `WatcherAtomics::init`) | `new_unchecked(heap::into_raw(Box::new(..)))` | `heap::alloc_nn(..)` / `heap::into_raw_nn(..)` | | a `Vec` that is given up (`Encoded::from_owned`) | `ManuallyDrop` and `new_unchecked(slice_from_raw_parts_mut(v.as_mut_ptr(), v.len()))` | `v.into_parts()` and `NonNull::slice_from_raw_parts` | | a `RefPtr` (two in `Bun.spawn`) | `new_unchecked(p.as_ptr())` | `p.as_non_null()` | - `heap::alloc_nn` and `heap::into_raw_nn` were themselves `NonNull::from(Box::leak(b))`. The documentation of `Box::leak` says to prefer `Box::into_non_null` for memory that is freed later, so they now call that. It compiles to the same pointer copy. - `ConcurrentTask::new`, which only `create` called, is folded into it. The FSEvents loop boxed a placeholder task and then overwrote every field through a helper with one caller. It now builds the task directly. - The job behind `crypto.randomBytes(n, callback)` kept a `JsPtr<u8>` and a length, and rebuilt the slice with `from_raw_parts_mut`. It keeps a `JsPtr<[u8]>`, like the scrypt job in the same file. - Almost all `new_unchecked` calls that remain start from a raw pointer: an FFI result, a `container_of` walk, pointer arithmetic, or the argument of an `unsafe fn`. #### 3. Four callback contexts borrow instead of holding raw pointers Four places build a context on the stack and pass it through a `void*` to a callback that runs before the call returns. Each kept raw pointers to other locals in that context, and the callback turned each pointer back into a reference in its own `unsafe` block. The context only lives for one synchronous call, so it can hold the borrows themselves. The other contexts of this kind are already written that way: `TablePrinter`'s `Ctx<'c, 'a>`, `URLSearchParams`' `Wrap<'a, Ctx>`, and the `ExpectedEntry<'a>` of `toBeOneOf` and `toContainEqual`. | Context | Used for | Held | Holds now | |---|---|---|---| | `AggCtx` | printing the members of an `AggregateError` | `*mut Formatter`, `*mut Writer`, `*mut ExceptionList` (null for none) | `&'c mut Formatter`, `&'c mut Writer`, `Option<&'c mut ExceptionList>` | | `ExpectedEntry` | `expect(iterable).toContain()` | `*const JSGlobalObject`, `*mut bool` | `&'a JSGlobalObject`, `&'a mut bool` | | a tuple | `AnyWebSocket::cork` | `(*mut C, fn(&mut C))` | nothing: `cork` takes the closure itself, as `Response::corked` does | | `Erased` | `PackageManager::sleep_until` | `*mut C` | `&'c mut C` | The one `unsafe` block left in each callback is the cast of the context pointer itself. A reference has the layout of a raw pointer and `Option<&mut T>` is a nullable pointer, so each callback does the same loads and the same null test. `cork` had one caller, which passed a closure and `|call| call()`, so each corked WebSocket event now makes one indirect call fewer. A comment above `cork` pointed to `NewWebSocket::cork`, which does not exist. It is removed, with the empty section heading of the same deleted type. #### 4. JS codegen workers borrow the linker, the chunk and the renamer with `&` Each part range of a JS chunk is printed by its own task on the thread pool. Every such task formed a `&mut LinkerContext` and a `&mut Chunk` from the shared raw pointers, so while a chunk is printed, many threads hold a `&mut` to the same two objects. The code said so itself: the worker's SAFETY comment noted that peer tasks hold their own `&mut` views "for read-only printer use", and the note on `unsafe impl Sync for Chunk` listed the renamer as an exception whose borrow "should become `&'r`". Nothing on that path writes through either object. The `&mut` came from a few signatures that asked for more than they use: - **`Renamer` held `&mut` to the underlying renamer.** All three `name_for_symbol` implementations only read. `MinifyRenamer`'s took `&mut self` because `TinyString::slice` did, under a comment carried over from Zig about the compiler copying the struct, which cannot happen behind a Rust `&self`. `Renamer<'a>` now holds shared references and is `Copy`. - **`should_remove_import_export_stmt`, `convert_stmts_for_chunk`, `convert_stmts_for_chunk_for_dev_server` and `generate_code_for_file_in_chunk_js`** took `&mut LinkerContext`, and two of them `&mut Chunk`. They take `&`. - **`module_preload_registration`** took both by `&mut`, next to a `&[Chunk]` that contains the same chunk. - **`Printer::mangled_prop_name`** took `&mut self`, and `post_process_js_chunk` borrowed the runtime's module scope with `&mut`. Both only look names up. With those fixed, the workarounds around them are not needed: | Where | What goes away | |---|---| | `Printer::name_for_symbol` | It erased the name's lifetime through a raw pointer, because a borrow of `&mut self.renamer` got in the way of the next `print` call. `Renamer::name_for_symbol` now returns `&'a [u8]`. | | `print_code_for_file_in_chunk_js` | A `transmute` of the `Renamer` to other lifetimes. The `&mut` inside made it invariant. | | `generate_entry_point_tail_js` | `detach_lifetime` on `import_records`, for the same reason, and an explicit `'a` on four parameters. | | `generate_compile_result_for_js_chunk_impl` | `chunk.renamer` split off through a raw pointer. | | `generate_code_for_file_in_chunk_js` | `parts` held as `*mut [Part]` and dereferenced in four places, `parts_live` held as a `BackRef`, and `parse_graph` read through the raw field, all to get around its own `&mut c`. `ExportHoist`, the context of a callback in that function, held `BackRef`s to the arena and to `c.graph.symbols`. It now borrows both. | | the JS and CSS workers | `c_ptr` and `chunk_ptr` dereferenced by hand. They use `ctx.c.get()` and `ctx.chunk.get()`, as the HTML worker already does, and `pending_part_range_prologue` no longer returns `c_ptr`. | | the CSS worker and `prepare_css_asts_for_chunk_impl` | A raw-pointer cast of `&c.graph.symbols` from `symbol::Map` to `symbol::Map`, once in each. In the first, also a `Vec` made with `from_borrowed_slice_dangerous` only to be passed as a slice. Without it the `Layers` and `ExternalPath` arms were identical, so they are one arm now. In the second, also `parse_graph` read through the raw field. | The old SAFETY comment in `Printer::name_for_symbol` said that the bytes always live in the AST arena or in the source. A minified name of up to 15 bytes is stored inside the renamer itself. The new signature ties the slice to the borrow of the renamer, which covers that case. Of the 17 blocks, 11 are removed outright. The other six were hand-written dereferences that now go through the existing safe accessors `ParentRef::get`, `BackRef::get` and `LinkerContext::parse_graph()`. Two things are left as they are: - For `--format=internal_bake_dev`, `convert_stmts_for_chunk_for_dev_server` still updates the file's own import records. It does that through its local `ast` view, not through `c`. In the dev server each file has one part range, so one task. `bun build --format=internal_bake_dev` with several entry points and no `--splitting` can print one file in two chunks, so there two tasks can still touch the same records. This PR does not change that. - `post_process_js_chunk` keeps `&mut Chunk`. It runs once per chunk and really writes to it. It also still takes `&mut LinkerContext` through `ctx.c()`, one task per chunk, so that later phase still has the aliasing that this change removes from printing. #### 5. Readers of the `FileSystem` singleton call `FileSystem::get()` `Transpiler` and `Linker` each kept a raw copy of the pointer to the process-wide `FileSystem`. Code that wanted the working directory or one of the string stores dereferenced a copy by hand. None of that code writes to the `FileSystem`. - **`Transpiler.fs` and `Linker.fs` are gone**, and with them `Transpiler::fs()`, `Transpiler::fs_mut()` and `Linker::fs()`. `fs_mut` gave out a `&mut FileSystem` with an unbounded lifetime from a `&self`. Its two callers, both in `process.chdir()`, only read the working directory. - **Readers call `FileSystem::get()`**, the existing shared accessor, as 45 other places on `main` do. That covers six sites in `bun run`, one in `bunx`, one in the bake router, `VirtualMachine::fs()`, `Scanner::fs()` and the former callers of the accessors. The sites that only wanted the filename store call `FilenameStore::instance()`. - **`bun create` and `bun upgrade`** turned the pointer that `FileSystem::init` returns into a reference, three times out of four a `&mut`. They only read, so they call `init` and then `get()`. `Example::fetch_all_local_and_remote` and `FileSystemTmpdirExt::tmpdir` take `&`, so the other two callers of `tmpdir` no longer need `FileSystem::instance()`, which returns a `&'static mut`. - **`BundleOptions::from_api`** took `&mut FileSystem` and passed `&mut fs.fs` through `init_external_modules` to `validate_path`, which ignored it. The parameter is gone from all three. - `wire_after_move` copied the pointer into the resolver again. `Resolver::for_worker` has already done that. The resolver keeps its pointer, because it really writes to the directory cache. So does the scanner of `bun test`, for `read_dir_with_name`. It copied `Transpiler.fs` and now asks `Resolver::fs()`, which returns the same value. No other crate reads the field `Resolver.fs` or the alias `fs::Implementation` any more, so both are `pub(crate)`. The compiler does not prove these reads safe. They now rest on the one `unsafe` inside `FileSystem::get()`, which is the accessor of a `static`, instead of on a separate `unsafe` at each site. In a release build `get()` is the address of that static, so no reader does more work than before, and `Transpiler` and `Linker` are each one pointer smaller. The readers that go through `FileSystem::instance()` are a much larger set and are left for a follow-up. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. The FSEvents loop is macOS-only, one half of the IPC setup is Windows-only, and `Encoded::from_owned` is compiled for macOS and Windows only, so for those three this type check is the only verification I could do. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/native-plugin.test.ts` | 19 | 0 | yes | | `test/bundler/bundler_plugin.test.ts` | 68 | 0 | yes | | `test/aot/build-plugins-native.test.ts` | 8 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/bundler/esbuild/tsconfig.test.ts` | 21 | 0 | yes | | `test/bundler/esbuild/css.test.ts` | 68 | 0 | yes | | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/js/node/fs/fs.test.ts` | 631 | 0 | yes | | `test/js/bun/spawn/spawn.test.ts` | 182 | 0 | yes | | `test/bundler/transpiler/transpiler.test.js` | 258 | 0 | yes | | `test/js/node/child_process/child_process_ipc.test.js` | 1 | 0 | yes | | `test/js/node/crypto/crypto-random.test.ts` | 27 | 0 | yes | | `test/js/bun/test/expect.test.js` | 427 | 0 | yes | | `test/js/bun/util/inspect-error.test.js` | 46 | 0 | yes | | `test/js/bun/util/reportError.test.ts` | 7 | 0 | yes | | `test/js/bun/websocket/websocket-server.test.ts` | 218 | 0 | yes | | `test/cli/install/bun-add.test.ts` | 220 | 0 | yes | | `test/bundler/bundler_cjs2esm.test.ts` | 113 | 0 | yes | | `test/bundler/bundler_merged_enum.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_minify.test.ts` | 67 | 0 | yes | | `test/bundler/esbuild/importstar.test.ts` | 99 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/esbuild/ts.test.ts` | 59 | 0 | yes | | `test/bundler/bundler_html.test.ts` | 26 | 0 | yes | | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | `test/cli/install/bun-run.test.ts` | 319 | 0 | yes | | `test/cli/run/run_command.test.ts` | 33 | 0 | yes | | `test/cli/install/bun-create.test.ts` | 139 | 1 | yes | | `test/js/node/process/process.test.js` | 194 | 4 | yes | | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | **Total** | **3778** | **43** | | The 43 failures are the same 43 tests on both builds: - 35 in `test/bake/dev` drive a browser page through the `happy-dom` package, and 1 in `resolve.test.ts` needs `tsyringe`. Neither package is installed on the machine I used. - 1 in `bun-create.test.ts` runs `bun create` with no template, which downloads the list of examples. - 4 in `process.test.js` check the ICU version, the host's Node.js version, `process.getgroups()` and `process.execArgv`. - 2 in `bun-test.test.ts` expect a `RangeError` for values nested too deeply to print. I also ran these on both builds. The output, 5,473 lines in all, is the same: - **Bundler (1, 2, 4):** 14 `bun build` commands over a small project, across formats, targets, minify flags, source maps, `--splitting` and `--format=internal_bake_dev`, and then five of the bundles. The output includes the bundles and the source maps. - **CSS (4):** bundles of a stylesheet that imports external URLs and local files with `layer()`, `supports()` and media conditions, and of a CSS module, plain and minified. - **Resolver (1):** `Bun.resolveSync` on 38 specifiers in a project with several `package.json` and `tsconfig.json` files, with and without `--preserve-symlinks`, then the same program through `bun build`. - **`NonNull` sites (2):** - sources that are plain, start with a UTF-8 BOM, are UTF-16 with a BOM, and are empty - `require` in a directory of 12,000 files, which reaches `BSSList`'s overflow block - recursive `fs.promises.readdir` - 200 async file operations in flight at once - 24 children at once with `BUN_FEATURE_FLAG_FORCE_WAITER_THREAD=1` - a child that talks to its parent over IPC, JSON and advanced, from `Bun.spawn` and from `child_process.fork` - `crypto.randomBytes` and `crypto.randomFill` with a callback - **Callback contexts (3):** - `AggregateError`s printed, reported, rejected and thrown - `toContain` on sets, generators and map iterators, passing and failing - `ws.cork()` in `open` and `message` handlers, with a return value, a throw and a `close()` inside the callback - `bun install`, `add` and `remove` with local tarballs - **`FileSystem` readers (5):** - `bun run` of scripts, files, `.`, local binaries, a system binary and missing names, from the project root and from a subdirectory - `bun x` of local binaries - `process.chdir()` that succeeds and that fails - `bun build --external` with relative, absolute and wildcard paths - `bun create` from local templates in three places - `bun getcompletes` - `Bun.FileSystemRouter` - `bun test` file discovery over files, directories and symlinks --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…compiler docs (oven-sh#47129) Follow-up to oven-sh#45631. Upgrades WebKit to oven-sh/WebKit `main` at f495db1d2488 (oven-sh/WebKit#930), which holds the engine's half of this work. Most of this is behind `--aot`. The first two sections list what is not. ### Changes that reach Bun without `--aot` - **`node:path`:** the native functions take the width of a substring's characters from its base's storage as it is now. `test/js/node/path/string-forms.test.ts` covers it. - **Heap snapshots:** `Bun.generateHeapSnapshot()`, its V8 form and the heap profiler release the profiler's snapshot once the JSON is built. - **`execve`:** the suspension lock is held over `execve`. - **`process.env` on Windows:** `Object.preventExtensions()`, `Object.seal()` and `Object.freeze()` throw a `TypeError`, as in Node. - **Module loader:** follows `VM::synchronousModuleQueue()` from the WebKit upgrade. - **Windows:** the crash handler registers with `WTFSetAbortReporter()`. - **Parser:** a class records whether `useDefineForClassFields: false` turned its fields into assignments in the constructor. `declare` is a name on the left of `for-of`, `for-in` and `for await`, and before every kind of line break. Any of the language's line breaks may follow `global`. - **`bun check`:** a ring of functions without annotations resolves in linear time. Only callable built-ins join a function type. The type of a well-known symbol is `symbol`. `--timing` prints the number of queries. ### `bun build --compile` - **`Bun.build({ compile })`:** `import.meta.main` is inlined for the entry point, as on the command line. - **Windows:** where the section table has no free slot for the payload's header, the sections' data is laid out again behind a table that has one. - **macOS:** the executable is built and signed over a private mapping of the template and written page by page, so the process never holds the whole file in memory. - **macOS:** a compiled executable no longer starts a background thread at boot to load the system CA certificates. They load on first use, on the thread that asks, as before oven-sh#45631. The flag for it, and the search of the output for `getCACertificates` that set the flag without `--aot`, are removed with their tests. - **`node:fs` on paths inside an executable:** the asynchronous calls on Windows name each path in an error as the caller wrote it. Asynchronous `statfs()` answers for the file system that holds the executable. - **`--trim`:** - On macOS, stack traces and the allocator's override know the several code segments of a trimmed executable. - The two TLS hooks that only the WebSocket client calls go with it. - On Windows, the tool tells a member of an archive from a file of the same name. - On Windows, a build on a trim template fills holes and joins pieces over them, so it cuts what a build on Bun cuts. ### The ahead-of-time compiler - **Modules:** - One set of `__esModule` marks serves both kinds of module. - Export tables of re-exporting files are limited like those of other files. - A name taken from a JSON import loads the files in between, so files that a package lists under `sideEffects` run. - A name that a library module lacks can be read off its namespace. - **CommonJS:** under `"type": "module"`, a default import holds what the link captured. It takes the short path only where nothing stores to `module.exports` after the body ends. - **Module names from lists and tables:** a walk of a list (`forEach`, `map`, `filter`, `find` and so on) counts as leaving the list unchanged only under these conditions: - its callback is written in the call - the callback has no rest parameter - the callback cannot reach itself through `arguments`, a direct `eval` or its own name - **Projects with a second `tsconfig.json`** take what the application's load found: `import.meta.main`, file identities, erased sites and folded constants. - **Classes:** - An unused class is removed whole only where its heritage is proven. - Methods stored on a constructor function's `prototype` are members of its layout. - **Workers:** - Whether the files and texts that Workers start from form one program with the rest is decided after the front end, before the engine's first run. - A Worker that finds no room for its thread waits for siblings that are ending, and is tried again on a timer. - **`node:http`:** only what `new ServerResponse()` makes takes the compact form. An instance of a subclass, and an object that `ServerResponse` is called on, keeps its state on itself. - **Build:** a third list of the engine's sources is compiled for minimum size. Dependencies that no code of a crate uses are gone from 13 `Cargo.toml` files. ### Docs `docs/compiler` is rewritten and about 5,800 lines shorter. Each page keeps a runnable example, which `test/cli/aot-docs.test.ts` runs. `docs/snippets/compiler-internals.jsx` had no importer left and is removed. The table of libraries known not to work moves next to its test. ### Tests and CI The ahead-of-time compiler's tests run in the ordinary `test-bun` shards. The `test-aot` and `test-aot-drawn` steps, their path triggers, draws and shares are gone, and so is every step that was allowed to fail. Every pull request gets the same pipeline. - **Lanes:** every test of the compiler runs on the two Alpine lanes. The other lanes run the ones that ask something of the system: processes, Workers, the environment, files, sockets, memory limits, the image, native code. Windows also runs the `.ir` tests. The lane with AddressSanitizer runs all but a list of files that feed it nothing new, on a copy of Bun without its debug information. The lists are in `scripts/runner.node.ts`, and a test file that says `// Runs alone:` stays out of the bucket of files that run side by side. - **Whole programs** are small files under `test/aot/parallel/`. Each passes when its executable exits and prints as `bun <file>` does, on Alpine also with its types erased. The runner runs them side by side, as it does Node's tests. - **Source to IR:** files under `test/aot/ir/` with `// RUN: lower` and `// CHECK:` lines. `cargo test` runs them in one Buildkite step. `AOT_IR_UPDATE=1` writes everything below a `// GENERATED:` line again. - **IR to IR, and IR to B3:** `.ir` files under `test/aot/engine-tests/`, run by `testaot` with Bun's table of built-ins. `bun test/aot/tools/update-ir-tests.ts` writes a file's input from its source and its expectation from its input. - **Servers, frameworks and packages** are ordinary test files: `test/aot/servers/`, `test/aot/frameworks/`, `test/aot/backend-packages/`. - **Frameworks:** Next and Angular compile to one file. Nuxt and SvelteKit compile with `--compile-in-place`. Each of these four has caught a compiler bug that no smaller test showed. The Astro and React Router apps never did, and building the site is most of what these tests cost, so those two apps are removed. The twelve units (six apps, two forms each) become four test files. - **Generators** moved to `test/aot/fuzz/`, where `bun test/aot/fuzz/run.ts` draws from them. No test file starts one. - The stored programs were cut to those that cover an edge of the compiler that no cheaper one covers, and those that came in with a fix. - `bun run rust:test` runs the Rust crates' unit tests in the same Buildkite step. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…7299) `cargo clippy` fails on `main`: ``` error[E0599]: no method named `fs` found for mutable reference `&mut transpiler::Transpiler<'_>` --> src/bundler/aot/testing.rs:84:39 ``` | PR | What it did | |---|---| | oven-sh#47273 | Removed `Transpiler::fs()` and rewrote its callers to `FileSystem::get()` | | oven-sh#47129 | Added `src/bundler/aot/testing.rs`, which calls `transpiler.fs()` | Each passed CI alone, and they merged one after the other. The file is behind `#[cfg(feature = "test_support")]`, so the build of bun is unaffected. The line now reads as oven-sh#47273 left the same field in `src/bundler/aot/mod.rs:119`. It is the only remaining call. Not compiled locally: this PR's `cargo clippy` is the check.
Keep bare plugin loading separate from filelike resolution and preserve lstat errors. Resolve Darwin paths through the canonical owner with its hard-link guard and ENOTDIR realpath fallback. Treat compiled namespaces like interpreted namespaces when CommonJS forwards their defaults. Preserve raw native-module mocks and align namespace, cycle and UTF-16 cache regressions with the paired engine. The real AOT forwarding case fails before the C++ adapter and passes after it through bun bd. Full Debug suites report 1201 passes and 12 timeouts under investigation; this commit does not claim full qualification.
Keep the kernel ENOTDIR result when the libc realpath fallback cannot run because process or system descriptors are exhausted. Successful fallback results and other errors retain their behavior, including Darwin trailing and intermediate path resolution. The unchanged native descriptor-exhaustion fixture reproduced file/below returning EMFILE before this repair. The preceding run passed the Node path oracle but had five POSIX lock-test timeouts after successful loop iterations. Fresh native qualification of this change remains pending; no passing full-suite result is claimed.
|
Hi! I'm the It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃 |
autofix.ci
Restore the default sealed namespace calls from a7c2bb6 after the upstream sync retained the mutable-namespace flag. The engine owns the separate require facade and preserves its live binding resolutions. The existing require-first, import-first and cache-first reflection cases reproduce the three failures at Object.isExtensible; keep their marker, cache and live binding assertions unchanged.
This was referenced Nov 2, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Merge the upstream Bun sync and repair the runtime contracts used by OpenClaw's upstream-synced WebKit engine. The signed series retains upstream merge parents and contributor history. Land with a merge commit; do not squash or rebase this series.
The follow-ups repair CommonJS namespace identity, resolver/VM contracts, filesystem forwarding, Darwin path-error handling, Node-hook resolution of plugin replacements, and require(esm) namespace sealing. Engine integration is in openclaw/WebKit#46; final qualification is in openclaw/WebKit#45. This replaces the conflicted older sync in #218, whose branch remains preserved.
How did you verify your code works?
These final runs use public Bun
30f563c446f5c0ebb2d71d16ac985c22c037b53b, tree5148dde801b8774f4b0fe610f310c89d708c75dd. Linux/downstream use engineb44e228; Darwin remains exact engineb801008evidence. The subsequent Linux-only cache-check correction leaves production engine and Darwin control inputs identical, verified before reuse. Source/control reviews are scoped clean.The earlier compiled CommonJS Debug proof and VM/AOT Debug proof remain scoped evidence with recorded timing failures. Darwin filesystem proof passes both focused path cases and seven contracts; its full Debug suite retains two POSIX-lock timeouts. This is not an all-green full Debug suite claim.
Remaining merge prerequisite
Stock CI needs immutable AOT Linux/macOS archives and the AOT source package absent from the unchanged shipping manifest. Lint/type workflows also need the matching engine source revision containing AOT generators. The older base's green CI does not request these assets and is not an inherited-failure waiver.
After separately authorized immutable artifact publication, promote one coherent manifest version/tag and complete asset checksums, add its changelog entry, and require fresh exact-head stock CI before merging. No release, tag, shipping-manifest promotion, or deployment has been performed by this PR's controlled qualification. The PR remains draft until those prerequisites are resolved.
Upstream handoffs use the existing namespace PR, Darwin hard-link PR, and synchronous-hooks PR, preserving authorship and distinguishing fork qualification from upstream qualification.