test: expand Node VM parity coverage

[TheHypnoo]

Summary

• expand the granular node:vm suite from 8 to 64 fixtures
• add 56 independent cases across API shape, contextification, execution, Script metadata/options, cached data, compileFunction, cross-context values, and gated VM modules
• add 25 follow-up contracts for property forwarding/descriptors, function/global bindings, cache views, repeat evaluation, evaluation errors, and top-level-await metadata
• document Node 26.5.0, Deno, and Bun primary-source selections plus concrete stopping evidence
• ratchet the measured VM baseline from 8/8 to 21/64 and update aggregate baseline totals, without changing runtime or compiler code

Upstream basis

Reviewed against:

• Node.js v26.5.0 (bebd1b8d92bf4cc917844d6335ed1ecf9c2a75fb): test/parallel/test-vm-*, VM ES-module tests, sequential VM tests, and doc/api/vm.md
• Deno main (c99e6904d8e297712ba859a64bbe848532d8f90f): tests/unit_node/vm_test.ts, tests/specs/node/vm_*, and ext/node/polyfills/vm.js
• Bun main (0ecd508247c7e99477717389a6cad44552cac023): custom VM tests, the vendored Node VM selection, and src/js/node/vm.ts

Each added contract was first checked on exact Node 26.5.0. The follow-up batches were repeated three times with identical output. The upstream compileFunction().toString() contract was also confirmed locally with Deno 2.9.2 and Bun 1.2.18.

Diagnostic results

The focused granular harness completes all 64 cases with no compile failures, crashes, timeouts, or skips:

Parity Pass:   17
Parity Fail:   47
Compile Fail:  0
Crashed:       0
Skipped:       0
Parity Rate:   26.5%

The independent baseline runner reports 21/64 passes and 43 diffs. The runner normalizes outputs differently from the focused shell harness; the committed floor uses only its measured result.

Stable mismatches expose:

• configurable export/constant descriptors and context-global identity
• incomplete deletion, descriptor, non-writable, symbol, own-key, and defineProperty forwarding
• missing function declaration/global binding behavior and fresh-context isolation details
• ignored context/Script/run options, missing microtask draining, and validation differences
• missing compileFunction() extensions, arguments, receiver, syntax classification, source representation, and option validation
• incomplete cached-data acceptance state and string filename metadata
• absent cross-context built-ins plus main-realm prototypes for structured values, descriptors, errors, and promises
• incomplete gated module constructor, syntax-error, evaluation-error, repeated-evaluation, lifecycle, and namespace semantics

context/code-generation-strings.ts and context/run-new-code-generation.ts are isolated as deliberate runtime-generated eval/new Function AOT exclusions. Direct static-source context, Script, and compileFunction mismatches remain genuine claimed-surface gaps.

Verification

PERRY_NO_AUTO_OPTIMIZE=1 ./run_parity_tests.sh --suite node-suite --module vm
# 17/64 parity passes, 47 stable output mismatches
# 0 compile failures, 0 crashes, 0 skips
# exits 1 only because the diagnostic suite is below the global 80% threshold

NODE_BIN=$HOME/.nvm/versions/node/v26.5.0/bin/node \
PERRY_NO_AUTO_OPTIMIZE=1 PERRY_RUNTIME_DIR=$PWD/target/release \
python3 scripts/node_suite_run.py $PWD/target/release/perry $PWD vm
# 21/64 passes, 43 diffs; exits 0

deno fmt --check test-parity/node-suite/vm
# passed, 66 files

bash scripts/check_file_size.sh
# passed

git diff --check
# passed

Exclusions and stopping evidence

The suite stops before dynamic import callbacks/loaders, network imports, inspector/debug breaks, signals, timeout races and infinite loops, escaped-promise timeout cases, WebAssembly code-generation execution, exact memory/cache bytes, GC/weakref/leak/stress cases, and engine-specific raw stack/message formatting. Context name/origin acceptance and validation are covered; their inspector-facing effects remain with inspector integration work.

A Node/Bun-derived Proxy-backed sandbox probe passes Node 26.5.0 but makes Perry exit with SIGSEGV; it is excluded from this deterministic print-and-diff suite and belongs in separate runtime crash coverage. Node 26.5.0 also aborts on the invalid compileFunction('', ['a-b']) parameter-name probe on the tested macOS build, so that unsafe oracle case is excluded while surrounding type validation remains covered.

Summary by CodeRabbit

• New Features
  • Expanded VM parity coverage across vm execution modes, Script/compileFunction behavior, cross-context value handling (including promises/structured values), and descriptor/export/introspection details.
  • Added validation and option-handling scenarios, plus microtask timing, DONT_CONTEXTIFY, cached-data and code-generation controls.
  • Extended experimental VM modules coverage (constructors, lifecycle, repeated evaluation, errors, top-level await metadata).
• Documentation
  • Restructured VM parity documentation with a clearer scope, coverage taxonomy, upstream evidence, and explicit testing boundaries.
• Tests
  • Updated suite baseline/recorded results to match the expanded parity tests.

[coderabbitai]

📝 Walkthrough

Walkthrough

Expanded Node vm parity coverage with standalone fixtures for APIs, contexts, execution, scripts, cross-context values, compile functions, and VM modules. Documentation now records coverage, upstream evidence, and AOT boundaries; baseline metrics were updated.

Changes

Node VM parity

Layer / File(s)	Summary
Coverage and stopping documentation test-parity/node-suite/vm/README.md	Documents the Node 26.5.0 oracle, fixture coverage, upstream selection evidence, stable comparison outputs, and excluded AOT categories.
API, context, and validation fixtures test-parity/node-suite/vm/api/*, test-parity/node-suite/vm/context/*, test-parity/node-suite/vm/validation/*	Adds export and descriptor inspection, context creation and property behavior, code-generation and microtask fixtures, and invalid-input validation cases.
Execution and compileFunction behavior test-parity/node-suite/vm/execution/*, test-parity/node-suite/vm/compile-function/*	Adds fixtures for execution isolation, lexical bindings, receivers, script runs, mutations, compileFunction arguments, extensions, metadata, syntax errors, and validation.
Script metadata and cross-context values test-parity/node-suite/vm/script/*, test-parity/node-suite/vm/cross-context/*, test-parity/node-suite/vm/metadata/*, test-parity/node-suite/vm/imports/*, test-parity/node-suite/vm/require/*	Adds script execution and metadata fixtures, cached-data coverage, import/require surface checks, and cross-context identity, descriptors, structured values, and promises.
VM modules lifecycle and baseline test-parity/node-suite/vm/modules/*, test-parity/node_suite_baseline.json	Adds module constructor, namespace, syntax, evaluation, repeated-evaluation, and top-level-await coverage; standardizes lifecycle error output and updates baseline counts.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• PerryTS/perry#5097: Updates parity baseline floors and related regression-guard tooling.
• PerryTS/perry#5166: Also adjusts node_suite_baseline.json pass and total thresholds.
• PerryTS/perry#5722: Covers related runInContext global identity and property-mutation behavior.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: expanding Node VM parity coverage.
Description check	✅ Passed	The description is detailed and covers scope, upstream basis, verification, and exclusions, even though it doesn't mirror the template exactly.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch test/expand-node-vm-parity

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test-parity/node_suite_baseline.json`:
- Around line 210-211: Update the top-level overall baseline metrics in
node_suite_baseline.json to include the vm increases: add 3 passes and 31 total
tests to the existing overall counts, then recalculate the overall percentage
from the resulting values.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a47e2d4e-78b6-4bf1-850f-ca5fa731e0ef

📥 Commits

Reviewing files that changed from the base of the PR and between 0043232 and 4dd5576.

📒 Files selected for processing (34)

• test-parity/node-suite/vm/README.md
• test-parity/node-suite/vm/api/descriptors.ts
• test-parity/node-suite/vm/api/exports-and-constants.ts
• test-parity/node-suite/vm/compile-function/arguments-and-receiver.ts
• test-parity/node-suite/vm/compile-function/context-extensions.ts
• test-parity/node-suite/vm/compile-function/validation.ts
• test-parity/node-suite/vm/context/code-generation-strings.ts
• test-parity/node-suite/vm/context/create-context-array.ts
• test-parity/node-suite/vm/context/create-context-shapes.ts
• test-parity/node-suite/vm/context/dont-contextify.ts
• test-parity/node-suite/vm/context/microtask-mode.ts
• test-parity/node-suite/vm/context/options-validation.ts
• test-parity/node-suite/vm/cross-context/builtin-identity.ts
• test-parity/node-suite/vm/cross-context/descriptors.ts
• test-parity/node-suite/vm/cross-context/promises.ts
• test-parity/node-suite/vm/cross-context/structured-values.ts
• test-parity/node-suite/vm/execution/context-lexicals.ts
• test-parity/node-suite/vm/execution/create-script-alias.ts
• test-parity/node-suite/vm/execution/function-receiver.ts
• test-parity/node-suite/vm/execution/run-in-context-mutations.ts
• test-parity/node-suite/vm/execution/run-in-new-isolation.ts
• test-parity/node-suite/vm/execution/run-in-this-bindings.ts
• test-parity/node-suite/vm/execution/script-run-methods.ts
• test-parity/node-suite/vm/modules/constructor-validation.ts
• test-parity/node-suite/vm/modules/context-and-namespace.ts
• test-parity/node-suite/vm/modules/lifecycle.ts
• test-parity/node-suite/vm/script/constructor-validation.ts
• test-parity/node-suite/vm/script/display-errors.ts
• test-parity/node-suite/vm/script/error-metadata.ts
• test-parity/node-suite/vm/script/run-options-validation.ts
• test-parity/node-suite/vm/validation/code-and-options.ts
• test-parity/node-suite/vm/validation/create-context.ts
• test-parity/node-suite/vm/validation/is-context.ts
• test-parity/node_suite_baseline.json

💤 Files with no reviewable changes (1)

• test-parity/node-suite/vm/modules/lifecycle.ts