Skip to content

bundler: detect a use directive after a comment or a hashbang - #42959

Open
robobun wants to merge 1 commit into
robobun/4def93c6/use-server-build-errorfrom
robobun/0f60dd35/use-directive-prologue-scan
Open

robobun wants to merge 1 commit into
robobun/4def93c6/use-server-build-errorfrom
robobun/0f60dd35/use-directive-prologue-scan

Conversation

@robobun

@robobun robobun commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • With server components, a comment or a hashbang before "use client" makes the module a plain server module. React builds then fail: "useState" is not available in a server component.
  • UseDirective::parse (src/ast/use_directive.rs:19) trims white space, then compares 12 bytes. A template literal and "use client".length also match.

Fix

  • scan_use_directive runs the lexer over the start of the file. The directive must be the whole first statement.
  • It returns the literal's range, so the "use server" error from bundler: report a "use server" module as a build error instead of aborting #42826 (this PR's base) points at the directive. UseDirective::parse and range are deleted.
  • With no framework (bun build --server-components), "use client" is now a plain module, so newly found files do not reach the first-line abort.
  • Verified: test/bake/use-directive.test.ts (new) and 3 devTests in bundle.test.ts fail on the base. Self-reviewed: 19 concerns raised, 15 addressed (Notes).

Background

  • React requires a use directive at the start of the file (comments are OK). The scan runs before the parse: its result picks the target.
  • Considered a byte scanner (a second tokenizer that can drift) and the parser's S::Directive (too late: the target is picked first).

Downsides

  • Behavior change: a comment or a hashbang no longer hides a use directive. Such a "use server" file is now a build error. `use client` and "use client".length stop counting.
  • A newly found module with export * or CommonJS exports gives undefined server exports, like its twin without the comment.
  • Cost: +6,195 bytes of text. With server components on, the lexer reads each file's leading comments and first token twice.
Notes

Origin. No user reported this. It came from the review of #42826, which keeps the byte prefix match and leaves this change to a separate PR. Alone on main, this change makes a "use server" file under a banner go from "bundled as a plain module" to the todo_panic! abort. #42826 turns that abort into a build error, so this PR sits on top of it.

What changes for a file (framework with server components)

Source starts with Before After
// comment or /* comment */, then "use client" server module client component
#!/usr/bin/env bun, then "use client" server module client component
BOM, U+00A0 or another white space the lexer accepts, then "use client" server module client component
comment or hashbang, then "use server" plain module build error (#42826)
a file that is only "use client" or "use server", with no byte after it plain module directive found
`use client` client component server module
"use client".length, "use client" + x, "use client" then .length on the next line client component server module
;"use client" client component server module
"use strict"; then "use client"; server module server module

A file read from disk loses its BOM in the reader. A source from a plugin's onLoad keeps it.

"use strict" then "use client" stays a server module on purpose. It is the shape that compilers emit for CommonJS. With separateSSRGraph: true the client reference proxy lists only ESM exports, so a CommonJS client module gives undefined exports on the server. A CommonJS file that starts with "use client" already behaves that way. Detection of the "use strict" shape would turn a working server module from a compiled package into undefined exports. The scan can walk the whole prologue when the proxy can list CommonJS exports. React's reference loader and Next.js accept the directive after "use strict".

bun build --server-components with no framework (bun build --server-components --target=bun entry.ts, entry.ts imports Comp.ts, release builds):

Comp.ts canary 1.4.3-canary.1+367d939d9 #42826 (49bc7cf) this branch
"use client"; on line 1 abort, exit 134 abort, exit 134 builds, plain module
comment or hashbang, then "use client"; builds builds builds, plain module
only the 12 bytes "use client" builds builds builds, plain module
"use server"; on line 1 abort, exit 134 build error build error
comment, then "use server";, or only the 12 bytes builds builds build error

Rows two and three are the reason for the guard in ParseTask.rs. Without it they abort on this branch (called `Option::unwrap()` on a `None` value). 5c5e077 on #42826 reverted a change to the old scan for the same reason. The guard also ends the abort of row one. #38046 (open) makes a directive with no framework a build error, because the bundle does not do what the directive asks for. This PR keeps such a build working, because these files build today. The choice is one place, the filter on the scan result in ParseTask.rs. If #38046 lands after this PR, it replaces the guard.

A newly found file takes the same paths as the same file without the comment. That includes the paths that do not work today. I measured these with separateSSRGraph: true (typeof Button on the server):

Module canary this branch Same file with "use client" on line 1, canary
banner, "use client", export { Button } from "./Button" object or function object object
banner, "use client", export * from "./Button" (Button.ts has its own directive) object undefined undefined
banner, "use client", exports.Button = ... (CommonJS) function undefined undefined

The last two rows are the proxy gap: it lists named_exports only, so export * and CommonJS exports are missing. A build error for every such module is not right: server code that uses only the named exports of a client barrel works today, and the client graph handles export *. The fix belongs in the proxy (pass export * through, list CommonJS exports).

Other paths that a client module can reach today:

The useState error has a second cause that this PR does not touch: a module with no directive that a client component imports is parsed for the SSR graph in a server mode.

Where the scan and the parser differ. The parser is more lenient than the specification in three places, and the scan follows the specification (as TypeScript, Next.js and React's reference loader do): ("use client"), "use client" as any, and a directive after a type-only statement such as type A = 1;. The directive text is compared raw, so "use\x20client" is not a use directive. React's loader compares the raw text too.

A source over 2 GiB. Every parser rejects one by length before it reads it (Source::check_parseable_len), because positions are i32. The scan does the same and leaves the error to the parser. Without the check, a source that opens a comment and never closes it ends in panic: int cast: TryFromIntError(PosOverflow). test/js/bun/transpiler/source-too-large.test.ts has the case.

Tests. test/bake/use-directive.test.ts checks 236 sources through the new scanUseDirective hook in bun:internal-for-testing, and one build with no framework. continues_expression_after_line_break sits next to parse_suffix and lists the arms of its match minus the three that stop at a line break (++, --, and the TypeScript !). The table test checks each of them, with LF and with CRLF. An earlier byte-level version of the same statement rule agreed with acorn's directive field on 200,000 random module sources (0 mismatches).

Cost, measured. Release builds of the base (49bc7cf) and of this branch (c39774b), same build tree, size bun-profile:

text data bss stripped bun file
base 80,858,372 110,424 1,822,992 81,000,008
this branch 80,864,567 110,424 1,822,992 81,004,104

Per process: one host function for the testing hook. Per parsed file with server components off: nothing, the scan does not run. With server components on and a JavaScript loader: the lexer reads the leading white space and comments, then one token, and a second token when the first is a use directive. No syscall and no file read. From the code, the scan does not allocate for an ordinary source. The lexer allocates when a leading comment starts with /*! or //! (it keeps a list of those) and when a token needs decoding (an identifier with an escape).

Time, release build. The scan alone through the testing hook: 2.1 µs for a 64-byte file with no comment, 3.2 µs for a file with a 1 KB banner. Those numbers include the call from JavaScript, a copy of the source and a new arena, which the bundler does not pay. A build of 2,000 modules with a 1 KB banner each, 15 interleaved runs for each binary, user plus system CPU time: base median 117 ms (96 to 137), this branch median 114 ms (99 to 149). The difference is inside the noise. Instruction counts are not measured: valgrind and perf are not installed on the build machine.

Suites run with the debug build: test/bake/use-directive.test.ts, test/js/bun/transpiler/source-too-large.test.ts, test/bake/dev/bundle.test.ts, production.test.ts, ecosystem.test.ts, plugins.test.ts and response-to-bake-response.test.ts in test/bake/dev, test/bake/app-options.test.ts, test/internal/source-lints.

Self-review items not done here

  • A diagnostic when the exports of a client boundary cannot be listed (CommonJS, export *). See above for why it is not a blanket build error.
  • CommonJS and export * support in the client reference proxy.
  • A parser diagnostic for a use directive that is present but not honored.
  • A measurement of how many files in published packages change sides.

[human-review] gate passed · iteration 5 · 10 files touched

fails on main (without fix)
ASAN without fix: 4 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/bake/dev/bundle.test.ts test/bake/use-directive.test.ts test/js/bun/transpiler/source-too-large.test.ts
bun test v1.4.3 (367d939d9)

test/bake/use-directive.test.ts:

# Unhandled error between tests
-------------------------------
SyntaxError: Export named 'scanUseDirective' not found in module 'bun:internal-for-testing'.
-------------------------------


test/bake/dev/bundle.test.ts:
Dev server testing directory: /tmp/bun-dev-test-9lfgy1
^[[0;30mdev|^[[0m Started development server: http://localhost:35523
^[[0;30mdev|^[[0m ^[[32mBundled page in 853ms^[[0m^[[2m:^[[0m routes/index.ts ^[[2m+ 1 more^[[0m
^[[0;30mdev|^[[0m [Bun] Hot update was not accepted because it or its importers do not call `import.meta.hot.accept`. To prevent full page reloads, call `import.meta.hot.accept` in one of the following files to handle the update:
^[[0;30mdev|^[[0m 
^[[0;30mdev|^[[0m Module "db.ts" is not accepted by routes/index.ts,
^[[0;30mdev|^[[0m ^[[32mReloaded in 90ms^[[0m^[[2m:^[[0m db.ts
^[[0;30mdev|^[[0m [Bun] Hot update was not accepted because it or its importers d
... (truncated)

release without fix: all passed
bun test v1.4.3-canary.1 (c39774be1)

test/bake/use-directive.test.ts:
(pass) use directive scan > finds a directive that is the first statement [2.82ms]
(pass) use directive scan > ignores a string that is not a directive [0.52ms]
(pass) use directive scan > follows automatic semicolon insertion after the string literal [2.62ms]
(pass) "use client" is a plain module in a build with no framework [45.95ms]

test/bake/dev/bundle.test.ts:
Dev server testing directory: /tmp/bun-dev-test-f27akX
^[[0;30mdev|^[[0m Started development server: http://localhost:36943
^[[0;30mdev|^[[0m ^[[32mBundled page in 5ms^[[0m^[[2m:^[[0m routes/index.ts ^[[2m+ 1 more^[[0m
^[[0;30mdev|^[[0m [Bun] Hot update was not accepted because it or its importers do not call `import.meta.hot.accept`. To prevent full page reloads, call `import.meta.hot.accept` in one of the following files to handle the update:
^[[0;30mdev|^[[0m 
^[[0;30mdev|^[[0m Module "db.ts" is not accepted by routes/index.ts,
^[[0;30mdev|^[[0m ^[[32mReloaded in 22ms^[[0m^[[2m:^[[0m db.ts
^[[0;30mdev|^[[0m [Bun] Hot update was not accepted because it or its importers do not call `import.meta.hot.accept`. To prevent full page reloads, call `import.meta.ho
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/bake/dev/bundle.test.ts test/bake/use-directive.test.ts test/js/bun/transpiler/source-too-large.test.ts
bun test v1.4.3 (367d939d9)

test/bake/use-directive.test.ts:
(pass) use directive scan > finds a directive that is the first statement [70.32ms]
(pass) use directive scan > ignores a string that is not a directive [124.12ms]
(pass) use directive scan > follows automatic semicolon insertion after the string literal [195.25ms]
(pass) "use client" is a plain module in a build with no framework [1333.17ms]

test/bake/dev/bundle.test.ts:
Dev server testing directory: /tmp/bun-dev-test-ii324h
^[[0;30mdev|^[[0m Started development server: http://localhost:42777
^[[0;30mdev|^[[0m ^[[32mBundled page in 775ms^[[0m^[[2m:^[[0m routes/index.ts ^[[2m+ 1 more^[[0m
^[[0;30mdev|^[[0m [Bun] Hot update was not accepted because it or its importers do not call `import.meta.hot.accept`. To prevent full page reloads, call `import.meta.hot.accept` in one of the following files to handle the update:
^[[0;30mdev|^[[0m 
^[[0;30mdev|^[[0m Module "db.ts" is not accepted by
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 4845ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/160] gen generated_host_exports.rs
generated_host_exports.rs: 121 exports (host=5, lazy=10, generic=106, rust=0); 248 extern-C blocks audited
[2/160] gen JS modules (bundle-modules)
Preprocess modules (24461ms)
Bundle modules (559ms)
Postprocesss modules (240ms)
Bundle Functions (988ms)
Generate Code (103ms)

[26.38s] Bundled "src/js" for production
  2598 kb
  197 internal modules
  13 native modules
  50 internal functions across 16 files
[3/40] rustc bun_ast 
[4/40] rustc bun_install_types 
[5/40] rustc bun_resolve_builtins 
[6/40] rustc bun_options_types 
[7/40] rustc bun_api 
[8/40] rustc bun_http 
[9/40] rustc bun_parsers 
[10/40] rustc bun_sourcemap 
[11/40] rustc bun_js_printer 
[12/40] rustc bun_react_compiler 
[13/40] rustc bun_ini 
[14/40] rustc bun_css 
[15/40] rustc bun_js_parser 
[16/40] rustc bun_resolver 
[17/40] rustc bun_router 
[18/40] rustc bun_bundler 
[19/40] rustc bun_transpiler 
[20/40] rustc bun_standalone_graph 
[21/40] rustc bun_bunfig 
[22/40] rustc bun_install 
[23/40] rust
... (truncated)
diff hotspot
src/ast/use_directive.rs                        |  51 -------
 src/bundler/ParseTask.rs                        |  24 ++--
 src/js/internal-for-testing.ts                  |   9 ++
 src/js_parser/parse/parse_suffix.rs             |  43 +++++-
 src/js_parser/scan/mod.rs                       |   1 +
 src/js_parser/scan/scan_use_directive.rs        |  38 ++++++
 src/runtime/api/JSBundler.rs                    |  35 +++++
 test/bake/dev/bundle.test.ts                    |  92 ++++++++++++-
 test/bake/use-directive.test.ts                 | 171 ++++++++++++++++++++++++
 test/js/bun/transpiler/source-too-large.test.ts |  18 +++
 10 files changed, 420 insertions(+), 62 deletions(-)

gate history · 1 passed · 2 rejected · iteration 5

evidence per changed file
file                                             reads  edits  tests
src/ast/use_directive.rs                             2      3     67
src/bundler/ParseTask.rs                             2      1     67
src/js/internal-for-testing.ts                       1      1     68
src/js_parser/parse/parse_suffix.rs                  2      0     67
src/js_parser/scan/mod.rs                            0      0     67
src/js_parser/scan/scan_use_directive.rs             3      6     67
src/runtime/api/JSBundler.rs                         1      2     67
test/bake/dev/bundle.test.ts                         4      2     44
test/bake/use-directive.test.ts                      0      1     36
test/js/bun/transpiler/source-too-large.test.ts      1      1      8

@robobun

robobun commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

Reproduced on canary 1.4.3-canary.1+09bb54630 with a framework app (Bun.serve({ app: { framework } }), the minimalFramework from test/bake/bake-harness.ts). A route imports Comp.ts and answers typeof Comp.Button. A client component gives object (a client reference). A server module gives function.

Comp.ts starts with canary this branch
"use client"; object object
// license header, then "use client"; function object
/* eslint-disable */, then "use client"; function object
#!/usr/bin/env bun, then "use client"; function object
`use client`; object function
"use client".length; object function
"use strict"; then "use client"; function function (on purpose, see the PR notes)

With the React framework, a component with a license banner before "use client" and a useState call fails bun build --app on canary: "useState" is not available in a server component. The same app builds on this branch.

The new tests fail on the base branch (#42826) and pass with the debug build of this branch:

bun bd test test/bake/use-directive.test.ts
bun bd test test/bake/dev/bundle.test.ts -t "use directive|use server"
bun bd test test/js/bun/transpiler/source-too-large.test.ts -t "use directive"

PR: #42959

@robobun

robobun commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

This comment no longer applies.

It explained two red checks (robobun/evidence and binary-size) that came from the old base: #42826 was behind the toolchain upgrade on main. #42826 now includes main, and this branch is rebased on its new head (c39774b). Both checks can run against this branch again.

@robobun

robobun commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:05 PM PT - Sep 29th, 2026

✅ @robobun, your commit c39774be1e6eb119600d77850fe08301887aa6d6 passed in Build #121679! 🎉


🧪   To try this PR locally:

bunx bun-pr 42959

That installs a local version of the PR into your bun-42959 executable, so you can run:

bun-42959 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline findings, I also checked the bun:internal-for-testing scanUseDirective binding (owned Source + local Arena, nothing arena-allocated owns heap memory, so no leak on the test path), the lexer-error-after-a-valid-directive case in scan_use_directive (returns None and the subsequent real parse reports the syntax error, so nothing is silently swallowed), and the Source hoist in parse_worker (same arena, same borrow of entry.contents, only moved earlier — no behavior change).

Extended reasoning...

The inline findings cover the behavioral widening (newly detected client boundaries hitting existing todo_panic!/CommonJS-proxy gaps) and the continues_expression table that must track parse_suffix. Separately I read the new scanner, the ParseTask.rs hoist of the Source allocation, and the test-only host function in JSBundler.rs: the host function builds an owned Source from to_utf8 and a fresh Arena that drops normally, the Log in the scanner is a plain local, and the hoisted Source construction is byte-identical to the deleted block with the same bump arena, so those paths do not add new memory or correctness risk. The ("use client") / "use client" as any divergence from the parser is called out in the PR notes as intentional spec alignment and is a design call for a human, not a defect in the scanner logic.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/bundler/ParseTask.rs
Comment thread src/js_parser/scan/scan_use_directive.rs Outdated
@robobun
robobun force-pushed the robobun/0f60dd35/use-directive-prologue-scan branch from 531a06a to afc53e8 Compare September 16, 2026 19:09
Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/js_parser/parse/parse_suffix.rs Outdated
Comment thread src/js_parser/scan/scan_use_directive.rs Outdated
Comment thread src/runtime/api/JSBundler.rs Outdated
@robobun
robobun force-pushed the robobun/0f60dd35/use-directive-prologue-scan branch from afc53e8 to 5afa20e Compare September 16, 2026 19:17
@robobun robobun changed the title bundler: detect a use directive after a comment, a hashbang or "use strict" bundler: detect a use directive after a comment or a hashbang Sep 16, 2026
@robobun

robobun commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Review follow-up for 5afa20e. Each thread has its own reply.

  • CommonJS ("use strict"; "use client";). The scan now accepts a use directive only as the first statement, after comments and a hashbang. That is the rule in the React documentation. The "use strict" shape is what compilers emit for CommonJS, and the client reference proxy cannot list CommonJS exports, so that shape stays a server module as on the base. I measured it on canary and on this branch. The PR body and the status comment have the new tables.
  • export * in a client module with a comment first. It now behaves like the same file without the comment (undefined for those exports on the server with separateSSRGraph: true). I did not add a blanket build error, because server code that uses only the named exports of such a barrel works today. The notes in the PR body give the measurements and the reason.
  • bun build --server-components with no framework. Not changed here. bun build --server-components: require the bun target and reject directives without a framework #38046 owns that path.
  • Token list. It moved next to parse_suffix as continues_expression_after_line_break.
  • Multi-line comments. All six are gone. The scan takes the file contents now, so the Source block in ParseTask.rs stays where it was and that file changes by 8 lines added and 9 removed.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

One verified lower-impact observation (a convention, logging or cleanup point) was not posted.

@robobun
robobun force-pushed the robobun/0f60dd35/use-directive-prologue-scan branch from 5afa20e to bb23eaf Compare September 29, 2026 20:11

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

…shbang

The scan that runs before the parse compared the first bytes of the
source with the quoted directive. A comment or a hashbang before
"use client" hid the directive, and the file was bundled as a plain
server module. A template literal, or a string that starts a larger
expression, counted as a directive.

scan_use_directive runs the lexer over the start of the file. The
directive has to be the first statement, and the string literal has to
be the whole statement. The scan returns the range of the literal, so
the "use server" error points at the directive and not at the start of
the file.

A use directive after "use strict" still does not count. That is the
shape compilers emit for CommonJS, and the client reference proxy
cannot list CommonJS exports.

`bun build --server-components` sets no framework. A "use client"
module is a plain module there, so the modules that the scan now finds
do not reach the abort that one with the directive on the first line
reached.

bun:internal-for-testing gets scanUseDirective for a table test of the
rules.
@robobun
robobun force-pushed the robobun/0f60dd35/use-directive-prologue-scan branch from bb23eaf to c39774b Compare September 29, 2026 21:39
@robobun

robobun commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Update for c39774b.

The PR body has the measured tables and the costs.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant