Skip to content

syntax.md

Latest commit

 

History

History
134 lines (101 loc) · 41.1 KB

syntax.md

File metadata and controls

134 lines (101 loc) · 41.1 KB
Raw

Syntax modules: parser, printer and minifier

Read this before touching the CSS, HTML or JavaScript parser, printer or minifier.

Shared by every language

lib/css/, lib/html/ and lib/javascript/ each hold the same five pieces:

  • syntax.js — a facade reaching syntax-parser.js (reads) and syntax-printer.js (writes) through getters, so a walk that never prints never loads the printer. It is neither parser nor printer.
  • syntax-parser.js / syntax-printer.js — algorithm only; every lookup table they read is in data.js — except generator-written regions such as the HTML entity table (// #region html entities in lib/html/syntax-parser.js) — generated by tooling/generate-css-data.js, generate-html-data.js or generate-js-data.js (see "Do not touch" in AGENTS.md).
  • A minify entry the default minimizer dispatches to: cssMinify.js, htmlMinify.js, jsMinify.js — and xmlMinify.js for .svg and .xml, under experiments.futureDefaults or an optimization.minimizeOptions.xml object.
  • A spec-conformance suite (yarn test:css-parsing, test:html5lib, test:xmlconf, test:test262) and the invariants and ecosystem comparison in tooling/COMPARE_TOOLS.md.

The node path

Every visitor and node printer receives one argument, the language's path: webpack.css.syntax, webpack.html.syntax and webpack.javascript.syntax hand over an object with the same members under the same names, named after ESTree where ESTree has the concept. NodePath<TNode> in lib/util/SourceProcessor.js is that contract, and SourceProcessor takes only a path satisfying it, so a missing or misnamed member fails lint:types.

const links = [];
new SourceProcessor()
	.use([NodeType.Element], (path) => {
		const href = path.fieldNamed(Field.attributes, "href");
		if (href !== 0) links.push([path.value(href), path.loc(href)]);
	})
	.process(source);
Member Returns CSS HTML JavaScript (ESTree)
node, parent, index the current node, its parent (null at a root), its position ✓ ✓ ✓
skipChildren(), stop() don't descend into this node; end the walk ✓ ✓ ✓
state what the run was given as process(src, { state }), typed by the processor's state type ✓ ✓ ✓
within(T) whether an ancestor of the current node has NodeType T a hash within a Declaration is a color a parent link walk inside a function, a loop, …
type(n) the language's NodeType member CSS Syntax names DOM names, plus Attribute type
range(n) [start, end] offsets into the parsed input, as acorn's range ✓ ✓ (an attribute's end past its quote) range
start(n), end(n) the same offsets as numbers, allocating nothing ✓ ✓ start, end
loc(n) [line, column, endLine, endColumn], lines from 1 and columns from 0 ✓ ✓ loc
source(n), source(s, e) the node as written; the input between two offsets ✓ ✓ ✓
name(n) its name, decoded; "" without one at-rule, declaration, function; a dimension's unit element, attribute, doctype, PI target Identifier.name
value(n) its value, decoded; "" without one token value: a number for a number, percentage or dimension; a simple block's bracket; text with escapes resolved otherwise text, comment, PI data, attribute value Literal.value
childCount(n), child(i, n) its children, one by one, 0 past the end a function's or block's value, a rule's prelude child nodes children in source order
children(n) every child as a new array, for a reader that keeps or searches them ✓ ✓ ✓
fieldCount(F, n), field(i, F, n) a Field's items, one at a time, 0 past the end; a single-node field has at most one prelude, value, declarations, rules attributes, content (single) arguments, callee, …
fieldNamed(F, name, n) the first item of a list Field whose name is name, 0 if none ✓ ✓ ✓
flag(F, n) whether a Flag is set important; integer (a number's type flag), id (a hash's) mathml, svg (an element's namespace), selfClosing computed, optional, …
rangeOf(P, n) where a Part is written, [start, end], null without one name, value (a url()'s contents), block name, value (inside the quotes), content, startTag, endTag, publicId, systemId —
startOf(P, n), endOf(P, n) the same as numbers, -1 without one ✓ ✓ —
textOf(P, n) a Part's text beyond the name and value, decoded; null without one none publicId, systemId raw, operator, kind, …
replace(text), remove() while printing, the current node prints as text, or as nothing ✓ (a declaration's ; optional) — —

Field, Flag and Part are exported beside NodeType and named with the spec's word: CSS Syntax's for CSS (a rule's declarations and rules read as empty on a block big enough to stream, so read those from the walk), the DOM's for HTML. A Flag is a bit of the node's flags column (important, an element's namespace and selfClosing) or read off a token's text (CSS's type flags).

path.fieldNamed(Field.declarations, "color"); // CSS: the declaration, 0 if none
path.flag(Flag.important); // CSS
path.field(0, Field.content); // HTML: a <template>'s fragment
for (let i = 0, n = path.fieldCount(Field.attributes); i < n; i++) {
	names.push(path.name(path.field(i, Field.attributes))); // HTML: nodes the shared members read
}
path.rangeOf(Part.name); // where the name is written, null without one

The types follow the same tables: a visitor registered for some node types gets a path whose field, fieldCount, fieldNamed, flag, rangeOf, textOf and within accept only the parts those types have, and an item read from a field is typed as its node type. So use([NodeType.Comment], (path) => path.fieldCount(Field.attributes)) fails to type-check, as does reading attributes off an attribute; a node of unknown type (parent, child(i)) takes any part.

No language has a member of its own, the writes below aside: what one grammar has is a row in its NodeType, Field, Flag or Part table, read through the members above, so a new language adds tables, not methods.

HTML's Part.content runs to where the content ends when the end tag was left out, and an element the parser inserted has no name, tag or content range.

skipChildren() still fires the node's exit, since the printer prints a node there. After stop() neither the current node's remaining visitors nor any pending exit fire. A print still writes the whole output, and a walk-only parse ends there: CSS once the top-level rule it was called in closes, HTML at once (HTML visits nothing before about 49k nodes have parsed, so stopping early in a big document still pays for that much parsing).

A run's state reaches every visitor as path.state, so visitors keep no closure of their own. Its type is the processor's: new SourceProcessor<{ count: number }>() in TypeScript, or a @type annotation in JSDoc, types path.state in every visitor and makes process take only that shape:

/** @type {import("webpack").css.syntax.SourceProcessor<{ count: number }>} */
const processor = new SourceProcessor();
processor
	.use([NodeType.Ident], (path) => path.state.count++)
	.process(source, { state: { count: 0 } });

One visitor serves several node types through an array, resolved once in use() so the walk sees the same per-type slots: processor.use([NodeType.Text, NodeType.Comment], visitor).

Conventions every language keeps, and a new member follows:

  • A node is an integer id, valid until the next parse; 0 is no node, and an HTML attribute's id is negative. The path is one object rebound before each callback; read it during the callback, never keep it.
  • A parse entry point returns ids too, read through the exported path A until the next parse (parseHtml, CSS's parseA*, JavaScript's parseProgram), and takes a parseError(input, code, start, end, severity) option reporting each parse error of the tokenizer and of the parser or tree construction: "error" where end of input cuts a token or rule short, or where a JavaScript one stops the parse, and "warning" otherwise; without the option, a "warning" goes unreported and parseProgram returns undefined for an "error". No spec names them all, so HTML's tree-construction errors take html5lib-python's names, JavaScript's take Babel's reasonCode kebab-cased (read off probes by tooling/generate-js-data.js), and CSS's are webpack's own.
  • The node is the last argument and optional, defaulting to the current one: path.range(), path.range(other), path.child(0).
  • A node has at most one name and one value; every other part is a Field, a Flag or a Part (its position, and its text when it has more than the name and value), so the generic members cover any grammar instead of one method per part.
  • name and value are what the language's spec defines, decoded as ESTree's Identifier.name and Literal.value are: escapes and character references resolved, a number as a number, a comment's text without its delimiters. source is exactly the input, so the spelling as written is source(path.startOf(Part.name), path.endOf(Part.name)). A node without a name or value answers "", a missing part null.
  • Positions come as numbers and as ranges, like acorn's start/end and range: start, end, startOf and endOf allocate nothing, so CssParser and HtmlParser read every position through them; range, rangeOf and loc ([line, column, endLine, endColumn]) are a fresh array per call.
  • Walk children one way everywhere: for (let i = 0, c = path.child(0); c !== 0; c = path.child(++i)). HTML keeps where the last child read stopped per parent, so this takes one step per child even with nested loops.
  • Nothing is stored for a read. A member reads the columns the parser already fills or derives its answer when called: HTML counts index along the sibling links and makes loc's converter on its first call in a parse. The walk writes no more than its position, so a read nobody makes costs nothing.
  • Writes act on the current node and only while printing. replace(text) and remove() take effect when the node prints after its visitors, which still run its printer for what it records; on a walk that prints nothing, or on a comment, which prints from the token stream, they do nothing, and a rule whose block is large enough to stream prints as written. In CSS, what holds a replaced node prints the text its children printed as, without the minifier's rewrites, which read fields the replacement no longer matches; the rest of the sheet minifies as before. HTML streams most elements to the output before their exit and JavaScript decides parentheses and spacing from the original node, so neither has writes yet.
  • Reads go through the path, and webpack's own plugins read through it. CssParser and HtmlParser use the members above and nothing else from a node, so what they do a plugin can; only the printers keep raw readers of their own (_rawName, _attributeList, …).

Generated tables

In the generator, derive — don't type out. Read tables from published datasets (mdn-data, color-name, @webref/idl) whenever derivable, including by analyzing a grammar rather than listing names: the value-definition syntax says which properties take an <integer>, so that set is computed. An existing SUPPLEMENT table counts as a source too — cosine at each eighth turn is sine two eighths along, and each inverse trig table is its forward one read back.

Per-construct behavior is a table too. Where the minifier differs per name (each math function; next, properties or at-rules), the per-name part is a descriptor in the generator and the shared part an engine in syntax-printer.js keyed by it. MATH_FUNCTION_FOLD is the example: it says how each function's arguments are read, which arithmetic runs and what unit results carry, so the printer implements none and names no function. The arithmetic is emitted alongside and bound by reference, not name, so an undefined name fails generation instead of folding nothing. A new function is one line; one whose arithmetic exists needs nothing else. A test must drive every descriptor (one input per entry) so a wrong-but-existing binding fails instead of silently declining. Hand-listing names into SUPPLEMENT is the last resort, each entry stating why it can't be derived (spec prose, an equivalence between spellings, a judgement no dataset states) — hand lists go stale unnoticed when specs move; derived ones turn a spec change into a reviewable diff.

What each generated table holds

  • lib/css/data.js — every table the CSS minifier looks names up in, plus the arithmetic its math-function descriptors bind to: from mdn-data + color-name (box shorthands, color-argument and math functions, named colors) and the generator's SUPPLEMENT of spec-prose tables and math primitives, by tooling/generate-css-data.js — which also holds the value-definition-syntax parser those grammars are read with, and runs generation only as the entry point so its tests can require it.
  • lib/html/data.js — every table the HTML parser and minifier look names up in: reflected-attribute tables from webref's HTML IDL (@webref/idl, @webref/elements) plus the generator's SUPPLEMENT and PARSER_TABLES of §13.2 tree-construction vocabulary, by tooling/generate-html-data.js — which also emits the // #region html entities block in lib/html/syntax-parser.js from the vendored tooling/html-entities.json (WHATWG's named character references).
  • lib/javascript/data.js — the JS parser's and printer's Unicode tables: run-length identifier ranges the tokenizer decodes at its first non-ASCII code point, and per-edition \p{...} property names (reached only for a pattern the engine rejected), which tooling/generate-js-data.js reads from the pinned acorn devDependency — bumping it moves them; plus the Unicode 8.0 identifier ranges the printer writes unescaped (so older engines read its output), read from terser's own frozen tables; and NodeType, Field, Flag, Part and NODE_FIELDS, each ESTree node type's number, fields and their slots in a SyntaxTree, typed out in the generator.
  • lib/javascript/parse-error-codes.js — each parse error message with its code, Babel's reasonCode kebab-cased, read off probes both parsers read; in its own module, loaded by the first error parseProgram reports, so a parse reporting none never loads it.
  • lib/javascript/syntax-printer-data.js — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (NATIVE_*, built only when nativeObjectTables() is called, so loading the module allocates none), each name with its unsafe / builtins_ecma >= N conditions in terser's order, parsed with acorn out of terser's lib/compress/native-objects.js. Plus the keyword, reserved-word and operator-precedence tables the printer names tokens by (parserTables()), parsed out of terser's lib/parse.js, and the DOM property names property mangling keeps (domProperties()), out of its tools/domprops.js. Plus the built-ins improve evaluates on literals (foldedBuiltIns()): tooling/generate-js-data.js places every member @mdn/browser-compat-data lists of Math, Number, String, Array and the global functions in FOLDED_BUILT_INS, FOLDED_CONSTANTS or, with the reason, UNFOLDED_BUILT_INS, and fails when a bump lists one in neither, or a folded one is missing, misplaced, or newer in Node than engines.node (folding it would tie the output to the build's Node). Plus the constructors improve calls without new, CONSTRUCTED_WHEN_CALLED, each one the spec has construct the same when called. Plus the built-ins improve drops (pureBuiltIns()): every global and standard static member BCD dates to engines.node, less the ones IMPURE_BUILT_INS and UNREAD_BUILT_INS name with the reason (eval, a promise combinator that may reject, Intl, …); a call is dropped only where running it at build time on its literal arguments does not throw.

CSS

  • syntax.js publishes the parser, the printer and SourceProcessor only — read a former re-export off parser or printer.

HTML

  • syntax.js mirrors css, publishing the parser, the printer and SourceProcessor only. syntax-parser.js holds the tokenizer, §13.2 tree construction and the entity table.

XML

  • parseHtml(…, { xml: true }) reads XML 1.0 through tokenizeXml and its own token handlers (XML_PARSE_CALLBACKS), so the HTML handlers carry no XML branch on their per-token path. Names are case-sensitive, there are no insertion modes, and the internal DTD subset's entities and attribute defaults are applied — an entity holding markup is tokenized in place and printed as its reference, under a nesting limit and an expansion budget.
  • The printer then writes XML, never HTML, and offers renderEmbeddedSource only an SVG or XHTML element's <style> (of no type or text/css) and style="" — the namespaces where they are CSS. xmlMinify.js hands those to builtinEmbeddedRenderer with the CSS options and browsers a .css asset gets.
  • An SVG data: URL a stylesheet or a document holds is minified the same way while the XML minimizer is on (svg in cssMinify and htmlMinify): builtinEmbeddedRenderer answers it only where that is shorter, quoting a tie with ' (xmlQuote) since the URL usually sits in ". An HTML <svg> subtree is offered with as: "foreign-element" and declined: it is HTML's to print. One in a style="" whose payload holds the quote the declaration list switches to is not offered yet.
  • yarn test:xmlconf holds both halves to the W3C XML Conformance Test Suite: each document's canonical form, and each printed output reading back to it.

JavaScript

  • syntax.js reaches parser and printer through getters: parsing never loads the printer, and the printer loads the parser for its parse phase, which reads an asset with it into a SyntaxTree the printer adopts. It is the only parser: a source it refuses is refused, and terser's parse options for what is not a program (parse.expression, parse.strict, parse.experimental_typescript) are refused too. toEstreeObject(tree, node, fields?) builds a SyntaxTree node's ESTree object, each call returning the same object: every field, or only those listed and the rest on a later call. It bridges hooks and plugins while the parser moves to that struct-of-arrays tree. The tree already reads as the node path does, a node passed last: type, start, end, range, name, value, field, fieldCount, flag and textOf with the NodeType, Field, Flag and Part that syntax.parser exports. SourceProcessor walks the tree a parse writes as the CSS and HTML ones walk theirs, each visitor handed that path and the children taken in source order (a template's quasis and expressions in turn). Asked for mode: "minify", the same walk prints: syntax-printer.js's printer returns each node's text from its children's, as minify prints a source it neither compresses nor mangles, comments left out and terser's format options read from format. minify prints such a source with it too where no source map is asked for and not every comment is kept (which printTree prints with fewer instructions), walking the tree its own parse wrote (walkSyntaxTree) with the comments it keeps, a name printed inside its parent's text where none of its own is read: each is taken as terser's printer reaches it, by the node holding its token, and printed once the text before it is, which a text opening with it leaves to the node printing that text. What terser's stream reads off the text before a node — has_parens before a function or object starting a statement, !--, </script — it reads off the tree instead, through two members only the JavaScript path has: tree, the SyntaxTree walked, and ancestor(level), a node enclosing the current one; and each child's text is read once (PrintContext#consume), so a long one is made flat and its pieces let go as it is printed.
  • syntax-parser.js — tokenizer, acorn-derived core and every production in one file (any parse reaches the productions). Given a syntaxTree, the lazy path writes each node into it as it finishes, an id in the tree's columns rather than an object; the few the parser reads back by name (a key, a module name, an exported declaration, a function before its body ends) stay objects until their parent is written, and an expression read as a pattern after all is retyped in place. Only the latest edition without parser plugins takes a tree, since acorn's own productions read their nodes back. regexp.js (pattern validator) is the one on-demand piece, reached only for a pattern the host engine rejected — never require it from a build path.
  • sourceType names which production a source is, for parseProgram, process and the printer's minify alike: "module" (the default), "script", or a FunctionBody read as acorn's commonjs goal reads one (return and new.target at its top, no import, export or import.meta): "commonjs", a CommonJS module in the function Node wraps it in, or "event-handler", an event handler attribute's value in the one HTML compiles it into, which a #! line cannot open. HTML offers only "event-handler" for an attribute, never "module". minify compresses a function's body inside a function built around its tree, so every rule a function's body reads by applies and a name it reads undeclared stays global (for a handler, the element's, form's or document's); it prints the body without that function, mapped where it was read, its #! line and comments kept, and a handler without the last semicolon the closing brace would leave off. parseProgram and process read import phases (import defer, import source) too.
  • syntax-printer.js — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; jsMinify.js is the minify the default minimizer dispatches JS to under experiments.futureDefaults, and goes through it alone; without that flag the minimizer dispatches to terserMinify as minimizer-webpack-plugin publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. createModules builds what terser's modules exported — the node factories (createAst), the parser's tables (createParseTables), the compressor and the output stream — and each entry of PHASES installs one part onto them: add phases there and nowhere else. A node is a plain object without methods, made by a factory createAst compiles from NODE_KINDS with its fields; its prototype is its kind's, holding its ESTree type, the values ESTree fixes for the kind (prefix, a name's role, a declaration's kind), the kind itself, which ast.kindOf reads, and its kind bits. A kind's prototype sits below a frozen prototype per terser class, chained as NODE_KIND_ANCESTRY gives each kind's classes, and createAst names a predicate per class, is<Class> (isLambdaNode), a bound isPrototypeOf of that class's prototype: the phases ask a node what it is through these, never instanceof. There are no node methods: every operation terser defined per class (optimize, evaluate, has_side_effects, drop_side_effect_free, size, scope analysis, …) is a function on ast dispatching on node.type through a table, a type absent reading the default, and walkers, transformers and the compressor are plain objects (createWalker, createTransformer, createCompressor), with ast.walkNode and ast.transformNode visiting children by type. A pure_funcs function is still handed a call under terser's field names (args, expression, print_to_string) through a read-only view. A parsed source is adopted from its SyntaxTree, reading the columns rather than calling a method per field (ast.adoptTree); an ESTree, which a parse.spidermonkey input is, through ast.adoptEstree, and ast.toEstree writes the SpiderMonkey JSON format.spidermonkey hands back, as terser's to_mozilla_ast did. correct wraps the methods with which terser writes a wrong program and changes only that output; corrections.enabled switches it off. improve writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there, at the top level too where toplevel owns the names of the vars it moves there, a number, string or bigint passed for a parameter nothing writes is read in place where shorter, a small function called from several places is copied into each call whose arguments it reads in order, a function called in place with only literals passed reads them in place of its parameters, in strict code an arrow called in place to run eval is the call it makes, (a, 0) && b is (a, 0) and (a, 1) && b is (a, b) (|| alike), typeof of a function or class declaration nothing writes is "function", a constant read in a template is written into its text where shorter, ! of an array or object literal is !1 after its effects, a binding of a function, not seen by other scripts, whose every read reads a value the writes its body runs before it leave known has each read written as that value and goes (void 0 and a value much longer than the name left alone), as does a primitive read off an object declared as a literal and read only through its properties, or straight off a literal, a function called straight off an object literal is called bare where it reads no this, a const object read only through its properties is split without reduce_vars, and a call of an exactly specified built-in on literals ("abc".charAt(1), Math.max(1, 2), parseInt("ff", 16), 100 .toString(16), [1].concat([2])) is replaced by its value, an array without holes as an array literal, and a Number constant by its number where shorter, and an expression nobody reads that only reads a built-in global (JSON, Math.PI, Object.prototype.toString), calls one on literals without throwing (new Set, Object.keys([]), new RegExp("a", "i")) or reads a global a typeof proved declared (typeof x != "undefined" && x) is dropped, as is what runs nothing: a statement of asm.js made only of a constant or a function, a label a break or continue reaching its loop or switch anyway names, an empty finally with no catch, ({} = value) of a value never nullish (leaving the value), a switch of constant tests whose cases only declare functions Annex B copies nowhere, and the constant a folded test leaves after its effects; and a shorter spelling of the same is written: Array and Object constructed as literals (new Array(1, 2) as [1,2]), new dropped where the call constructs the same (Error, Function, RegExp passed no regular expression, …, by CONSTRUCTED_WHEN_CALLED), a spread array literal passed as its elements, typeof x == "undefined" as typeof x > "u", a + beside a number literal in arithmetic dropped, !(a == b) as a != b, !! of a boolean dropped and Boolean(x) as !!x, ?? after a value never null or undefined dropped, an integer tested against zero as the integer, === between numbers as ==, an array literal or a bigint concatenated as the string it makes, a string literal's character ("abc"[1]), +[], Number, String and BigInt of a literal and toString of a boolean or a string as their value, c ? a = x : a = y as a = c ? x : y (a name no with reaches, an undeclared one in sloppy code only, a property of this where it is bound or of a name nothing reassigns), (a = b) === null || a === void 0 as null == (a = b), from ECMAScript 2020 a == null ? void 0 : a.b as a?.b (and null == (a = b) ? void 0 : a.c as (a = b)?.c), and from ECMAScript 2021 a || (a = b) as a ||= b (a name or a property of a name assigned, && and ?? alike) and a = a || b as a ||= b (a var, let or parameter, which writing back to itself shows nothing), a name in a scope a with reaches is mangled where no with body spells it, and const as let where nothing writes the name, below the top level and where no eval or with reaches (once names are mangled, so the letters mangling counts stay terser's); collapse_vars also folds a value into a plain = write of a name another function declares, into a store to a property of a let or const declared before it, and into what a for…in or for…of whose head declares no let or const loops over, and a bare var after a for with an empty head is declared in that head; a function's last return c ? void a : void b runs c ? a : b and falls off, a last if (c) e there is written c && e, and a statement before a for…of stays out of its head, where the sequence would need parens; a module's imports of one module merge where they fit (import{default as a} as import a), as do its export{…} lists, a quoted name another module knows written bare where it is one (import{"a"as b}), and function f(){}…export default f (or a class the export follows) is declared as that export; a var without values is declared in a later var of its list past statements declaring nothing, or in the head of a for…in or for…of of the next statement assigning it, in the same function, that holds its every read; a number multiplied or masked (*, &, |, ^) is written last, as an equality's null or number is; drop_unused also drops from a declared or parameter pattern the names nothing reads (an array's as holes but for the last, which goes as one only where a default makes it longer, an object's where no rest follows and no getter could run: under pure_getters, or reading a literal), a declaration whose object pattern then names nothing and reads a literal (an array pattern still looks up its value's iterator), and from ECMAScript 2019 a catch binding nothing reads — taking built-ins as present and as the language defines them, as oxc and swc do —, and from ECMAScript 2015 a compressed program whose every string reads the same as a template of its length (none a directive, a quoted property name, a module's name or attribute, a tag, after await or yield, passed to require or Object.defineProperty, which Node's cjs-module-lexer reads quoted, or holding a backtick, ${ or both quotes) writes them all with backticks, which deflate codes shorter than a double quote, under quote_style: 0 with semicolons, a mix measuring bigger (improvements.backticks.enabled switches that one off); and improvements.enabled switches it off. The parse phase builds the tree from syntax-parser.js; javascript/printer.unittest.js holds the adopted tree to terser's own conversion, and PARSED_DIFFERENTLY in javascript/printer.unittest.js lists where the tree it builds prints otherwise than terser's. test/specCases/minify-corpora.spectest.js holds the whole printer to terser's output, a refusal matching any refusal, over each corpus in its CORPORA list (add a new one there) under the option sets in OPTION_SETS, and runs each output whose test states what its input prints (terser's expect_stdout, swc's expected.stdout and exec.rs) in terser's sandbox. A byte difference is accepted only where switching the correct and improve phases off gives terser's bytes back, and fails where improve writes more than without it, unless the program is shorter and only const written as let makes it gzip bigger (improvements.lets.enabled switches that one off to tell); what it fixes is listed in CORRECTED (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in INHERITED, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in REFUSED_BY_REFERENCE with what makes the source legal; where webpack refuses a source the reference reads, the source is listed in REFUSED_BY_WEBPACK with the rule it breaks (a test262 test expecting a parse error needs no entry); where it reads or prints one differently, the case is listed in REFERENCE_MISPRINTS with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's output.js or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless test/helpers/swcSmaller.js lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by oxc-minify, held the same way to test/helpers/oxcSmaller.js; an oxc output that misprints is no lead. JS_MINIFY_REPORT=<file> tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the terser devDependency — bump both together, and regenerate data.js.
  • Printing and mangling read the compressed tree as it is held, by its ESTree types: no conversion step stands between the compressor and the printer, so a tree a caller passes in or format.ast hands back is printed and mangled as it is. Where the compressor holds a shape of its own the printer reads it directly — a template's segments, two side by side printing as one text; an import's imported_name and imported_names, an export's exported_definition, exported_value and exported_names, attributes as the object holding them; import()'s args; a key's or a property read's name as a symbol node; a literal's role for a regular expression or a bigint's digits — and a name prints as its definition names it (mangled_name, else the definition's own name), terser's or the mangler's alike, never renamed in place. Where nobody keeps the tree, analyzeManglingScopes and printProgram drop what only the compressor read (COMPRESSOR_ONLY_FIELDS) off each node they read; every field written is one its kind's factory made. The estreeMangle phase mangles that tree as terser's mangler did: analyzeManglingScopes resolves its scopes, writing each name's definition, computeEstreeCharFrequency orders the identifiers, mangleManglingScopes gives the definitions their mangled_name and renameMangledEstree renames #private names and properties in one walk, skipped where there are neither. printEstree (the estreePrint phase) dispatches on node.type through three tables — codegen (the text), needsParens and sourceMapping — and asks about a parent through the kindOf predicates, never instanceof; the stream (MinifiedOutput) reads comments through NodeAccess. Everything that prints goes through it: minify, the mangler's character frequency and the compressor's printToString.
  • yarn test:minify-corpora — webpack's JS minifier vs the published one it replaces, byte for byte (a refusal matching any refusal), over every JS corpus: terser's own test/compress and test/input, test262, swc's minifier tests and oxc's; where a test states what its input prints, the output is also run and must print it. Each output must also be no bigger than swc's recorded one unless SWC_SMALLER lists why, and every output outside test262 no bigger than oxc-minify's unless OXC_SMALLER does; JS_MINIFY_REPORT=<file> tallies worse, the same and better.