Repository navigation
Conversation
The Lifetime::Clone arm of to_array_buffer_view_with_bytes detached the Blob before it threw the out of memory error. Blob.prototype.bytes() calls that arm on the caller's own Blob, so a bytes() that was refused left that Blob empty while its size stayed. The callers that pass a private copy to the arm release it themselves.
|
Status: draft until #46528 is merged, then cut again on top of it (decided there). The change is complete. Reproduced on a release build of main 179664e with no flags: Tests: 6 of the 8 new cases in CI: build 132359 finished with 180 of 181 jobs passed. The red job is one of the 2 shards of darwin aarch64. It hit its time limit after |
|
Updated 8:46 PM PT - Oct 24th, 2026
❌ @robobun, your commit f26df40 has 2 failures in
🧪 To try this PR locally: bunx bun-pr 46835That installs a local version of the PR into your bun-46835 --bun |
Draft until #46528 is merged, then cut again on top of it (decided there). See "Landing order" in Notes.
Problem
bytes()on an in-memory Blob of 2^32 bytes or more rejects withRangeError: Out of memoryand empties the Blob. ThenBun.write(path, blob)writes 0 bytes,new Response(blob)servescontent-length: 0, andBun.inspect(blob)prints[Blob detached].sizestays.self.detach()in theLifetime::Clonearm ofto_array_buffer_view_with_bytes(src/runtime/webcore/Blob.rs:2696).Blob.prototype.bytes()runs that arm on the caller's own Blob.Fix
bytes()leaves the Blob,Fileor slice as it was.ByteBlobLoader.rs:125,Blob.rs:2774,Drop for S3BlobDownloadTask.test/js/web/fetch/blob-oom.test.ts, 8 new cases, 6 fail on main. Alsoblob.test.ts,body.test.ts,streams.test.js.Background
Store.detach()drops the Blob's store reference and keepssize.Lifetime::Clonecopies the bytes.Lifetime::Transfermoves the store out of a Blob that a body owns.blob-oom.test.ts. Handle OOM better #13142 added it and gives no reason.Blob.prototype.bytes(): the arm would still empty a Blob that it was lent.Downsides
bytes()runs the same 691 instructions..textstays 82,164,617 B.The trade, for the reviewer to decide. #13142 added the detach to the
.cloneand.transferarms together and gives no reason for the.cloneone. If it was there to free the store at the throw, this PR gives that up. The rows are release builds with no flags.bytes()has rejected an in-memory Blob of 2^32 + 3 bytes.Bun.write(path, blob)new Response(blob)fromBun.servecontent-length: 0content-length: 4294967299Bun.inspect(blob)[Blob detached]Blob (4.29 GB)bytes()RangeError: Out of memoryNotes
Line numbers are those of main 179664e. "Release" means
--profile=releasebuilds of main and of this branch. "Debug" means the ASAN debug builds of the same two trees.Landing order
match. ItsBlob.rs:2689readsLifetime::Clone | Lifetime::Transfer => self.detach(),.Clone. The 6 cases here then fail.if LIFETIME == Lifetime::Clone { self.detach(); }.Reproduce
No flags. It needs about 5 GiB of RAM and 4 GiB of disk.
bytes()blob.sizeBun.inspect(blob)[Blob detached]Blob (4.29 GB)blob.slice(-3).text()"""XYZ"blob.stream(), bytes readBun.write(path, blob)over a file that existsnew Response(blob)fromBun.servecontent-length: 0content-length: 4294967299, all receivedblob.bytes()RangeError: Out of memoryblob.arrayBuffer()RangeError: Out of memory, as for a new Blob of this sizeThe Blob does not have to come from
new Blob(). Afetch()response of 2^32 + 1 bytes (4,176,021 bytes of gzip on the wire) read with.blob()gives the same: after the rejectedbytes(),Bun.write(path, blob)writes 0 bytes on main and 4294967297 bytes here.A cheap form of the same path: a 500,000 byte Blob with
BUN_GARBAGE_COLLECTOR_LEVEL=0 BUN_FEATURE_FLAG_SYNTHETIC_MEMORY_LIMIT=100000. Bun reads the second variable only when the first is set.Which reads empty the Blob on main
Measured on main with a new Blob per row, above the limit.
blob.bytes(), also on aFileand on a sliceblob.text(),blob.json()new Response(blob).bytes()blob.stream().bytes(),Bun.readableStreamToBytes(blob.stream())The callers of the
ClonearmFour routes reach the refusal in the
Clonearm. Only the first passes a Blob that a script holds.Blob.prototype.bytes()(Blob.rs:1173): the caller's Blob. This is the bug.Any::to_action_valuefor a stream read (Blob.rs:6035):ByteBlobLoader::to_buffered_valuebuilds a private Blob from the stream's store reference and callsblob.detach()after the read (ByteBlobLoader.rs:125).Transferarm for a store with more than one reference: it callsself.detach()after theClonearm returns (Blob.rs:2774), and the body callers detach again (Body.rs:1120,Body.rs:2014).Blob::dupeofS3BlobDownloadTask, andDropreleases it.RefPtr<Store>also releases its reference onDrop, so a private copy cannot leak the store.History
.cloneand.transferarms oftoArrayBufferViewWithBytesinblob.zig. The PR body, its review threads and the commit give no reason for the.cloneone. The Rust port kept it.blob-oom.test.tsworked around it: eachBlobcase builds its own Blob, and a comment named the detach. This PR deletes that comment and leaves the four cases as they are.bytes()part of Blob: keep the caller's Blob intact after a slice read, an array body, and a rejected bytes() #38503 (commit 5314e5d), which was closed to split it. Blob: keep text() and json() as UTF-8 after an ASCII prefix slice was read #46719 and Blob: share a body part's store instead of moving it out of the caller's Blob #46732 are the other parts.Tests
test/js/web/fetch/blob-oom.test.ts,describe("Blob after .bytes() rejects"), in-process under the 4 MiB limit of the file:File, and a slice that holds the last reference to a larger store. After twobytes()calls, onetoEqualcovers both rejections,size,arrayBuffer()(length and content), the first and last 3 bytes, andnew Response(blob).arrayBuffer(). 4 cases, all fail on main.bytes()resolves at the limit and rejects one byte above it, and the Blob above the limit keeps its bytes. Fails on main.blob.stream().bytes()andBun.readableStreamToBytes(blob.stream())reject and leave the Blob intact. 2 cases. They pass on main too. They hold the stream route, which shares theClonearm.heapStats().objectTypeCounts.Blob). Fails on main at thearrayBuffer()check.Measurements
bytes(): size 9437184,arrayBuffer().byteLength9437184 (main: 9437184, 0). Blob cells before / alive / after drop andBun.gc(true): 0 / 1 / 0 in 100 of 100 runs, the same as main (heapStats().objectTypeCounts.Blob, the Blob held only as a property, debug).new Blob/ after the refusedbytes()with the Blob held / after drop and gc: 0,1,1,0 in 100 of 100 runs (main: 0,1,0,0). Debug.Blob::detachcalls from theClonearm per refusedbytes(): 0 (main: 1). Store destroyed inside thebytes()call: 0 (main: 1). Store destroyed by the finalizer of the Blob after drop and gc: 1 (main: 0). gdb breakpoints, debug.to_array_buffer_view_with_bytes::<Clone, Uint8Array>: 390 B -> 365 B. The success path has the same instructions: the load of the store moves below the limit compare and uses another register. TheArrayBufferinstance stays 326 B..text: 82,164,617 B -> 82,164,617 B (llvm-nm -S,llvm-objdump,llvm-size, release).blob.bytes()on a 12-byte Blob, host function entry to return: 691 -> 691 on each of 16 warm calls. The four calls before them are 1252, 1252, 1252, 2579 on both builds (gdbstepi, JIT off, release).blob-oom.test.tswith the settings of the ASAN lane: 0 (main: 0). After a refusednew Response(blob).bytes()/blob.stream().bytes()/Bun.readableStreamToBytes(blob.stream()): the caller's Blob has 9437184 bytes each, memfd stores after drop and gc 0 each (main: the same).blob-oom.test.ts: 8 new in-process cases. 6 fail with the 1.4.3 canary and with a release build of main, 0 fail here. 800 of 800 pass with--rerun-each 100, debug and release.Suites run on the debug build of this branch
blob-oom.test.ts(30 pass, 1 skip),blob.test.ts(149),blob-cow.test.ts(5),body.test.ts(776 pass, 2 skip),body-clone.test.ts(100),blob-write.test.ts(15),response.test.ts(23),streams.test.js(727 pass, 2 skip),readable-stream-blob-consumed.test.ts(1),streams-string-limit.test.ts(9).Not in this PR
Lifetime::Transfer. They run on a Blob that a body owns.Blob::getthat empties the Blob ofnew Response([blob]): Blob: share a body part's store instead of moving it out of the caller's Blob #46732.bytes()still rejects a Blob of 2^32 bytes or more, and that includes exactly 2^32 bytes, a size that aUint8Arraycan have. new ArrayBuffer larger than 4 GiB throws RangeError: Out of memory (Node allows it) #46531 asks for a higherArrayBuffercap.