Repository navigation
chore: standardize the repository like tree-sitter-kotlin - #6
Merged
Merged
Conversation
Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
ECMAScript never inserts a semicolon inside a for header, but the grammar accepted an automatic semicolon after its declarations. That made `for (let x\n of y)` ambiguous until `of`, so an incremental reparse could reuse a declaration built when the ambiguity had been resolved the other way, and `for (var x\n of y)` failed to parse. Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
…e CLI Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
…kind field Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This repository was imported with upstream's setup: npm with ESLint, the C/Go/Node.js/Python/Swift bindings, and upstream's CI, fuzz, and lint workflows, with no release pipeline. WillBooster/code-gauge needs a Rust crate of this grammar that it can pin to a release, as it does with WillBooster/tree-sitter-kotlin. That requires the same tests and release pipeline as that fork.
Requirements
grammar.jsin the npm package, since WillBooster/tree-sitter-typescript will build on it.&and the tagged templates with invalid escapes (e.g.String.raw) unfixed here; they get their own issues.[after an arrow function with a block body (examples/cli/lib/base-cmd.jsinscript/known-failures.txt) unfixed here; the requester files its issue.for_statementinitializer only with;, because ECMAScript inserts no semicolon inside a for header and accepting one there madefor (let x\n of y)ambiguous, which broke incremental reparsing.LICENSE, as tree-sitter-kotlin does, without naming the upstream authors as authors of this fork's packages.@willbooster/tree-sitter-javascript, the cratewillbooster-tree-sitter-javascript), so nothing competes with upstream'stree-sitter-javascriptpackages.LANGUAGEandNODE_TYPES), the two forms the Kotlin fork ships and its consumers use.examples/, againstscript/known-failures.txt, so the list of failures changes only with the grammar.queries/in the repository, untested and not referenced fromtree-sitter.json, because the test layout allows only corpus and unit tests and neither package ships it, so the shippedtree-sitter.jsonmust not name query files absent from the npm package and the crate (the alternative, shipping untested queries, was rejected); the Kotlin fork's metadata names no queries either.Customer Summary
Nothing is released yet. After this merges, the first
feat/fixcommit releases@willbooster/tree-sitter-javascriptto npm (withtree-sitter-javascript.wasmandgrammar.js) and thewillbooster-tree-sitter-javascriptcrate to crates.io, as version 1.0.0.Technical Summary
.releaserc.json, thetest/test-rust/robustness/release/semantic-prworkflows,script/,test/unit/,bindings/rust/,Cargo.toml,Cargo.lock, andpackage.json.package-lock.json,eslint.config.mjs, the non-Rust bindings and their build files, upstream'sFUNDING.yml, issue templates, dependabot config, and workflows, and the highlight and tag tests intest/highlightandtest/tags.tree-sitter.jsonkeeps upstream's grammar entry without the query paths, since neither package shipsqueries/, names WillBooster Inc. as the author, and enables only the Rust binding.grammar.jsis reformatted by oxfmt and made lint-clean:() =>for unused$,\u0000-\u001Finstead of\x00-\x1F, an unnecessary\-removed, and theuflag on the regexes using\p{…}. At the first commit, the regeneratedsrc/parser.cis byte-identical to the one generated from upstream'sgrammar.js, andsrc/grammar.jsondiffers only in those regex spellings and flags. What to check hardest: the\[escapes in the regex-pattern and identifier character classes stay with lint suppressions, because the autofix removed them and tree-sitter's regex parser then rejects the pattern.src/differs from whatgrammar.jsgenerates, since both packages ship the committedsrc/.test/unit/performance.test.tsalso rejects a Wasm build compiled from asrc/parser.colder thangrammar.js. Renovate updates the fuzz job'stree-sitter-versiontogether withtree-sitter-clithrough a regex custom manager inrenovate.jsonc. oxlint and oxfmt ignoreexamples/, whose JavaScript is test input: the autofix had rewritten it.let jinsidefor (, both a declaration ended by an automatic semicolon and a for-in header ended by an optional automatic semicolon were live, sofor (let j\n of k)stayed ambiguous untilof. After an edit that turnedof kinto something else and its undo, the reparse reused thelexical_declarationbuilt when the for-in path had died, and yielded an ERROR. ECMAScript never inserts a semicolon inside a for header, so the declarations in afor_statementinitializer now end only with;(for_lexical_declaration/for_variable_declarationrules used only through aliases to the public node names, so the node types are unchanged; they are visible rules because a hidden one would makefor_statementinherit theirkindfield), and_for_headerno longer accepts an automatic semicolon. The ambiguity and its GLR conflict are gone, andfor (var j\n of k)now parses, where it used to fail; the corpus case covers it. What to check hardest: that the 1,640 example files parse to the same trees as before the fix (checked), and that rejectingfor (let i = 0\n i < n; i++), which is not valid JavaScript, loses nothing real-world code needs.src/is regenerated with tree-sitter-cli 0.27 (upstream used 0.25).Testing
bun run verify,bun wb test(corpus 116/116 with the native and the Wasm build, incremental-parsing fuzz, performance, and examples), andcargo test --lockedpass locally.tree-sitter fuzzpasses on all corpus cases with seeds 1, 2, 3, 4, 5, 42, and 1234 (1,000 iterations, 10 edits) and with seeds 7 and 99 (20 edits).script/parse-examples: 1 of 1,640 files fails,examples/cli/lib/base-cmd.js, which is valid JavaScript: a line starting with[after an arrow function with a block body is parsed as a subscript of that arrow function instead of a new statement.main), the macOS Intel and Linux arm64 jobs, and libFuzzer (Apple clang has no libFuzzer).Notes
queries/is kept but no longer tested (neither package ships it).semantic-pr / semantic-prruns onpull_request_target, which uses the workflow files onmain, so it cannot run for this PR.