Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions scripts/node_suite_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,25 @@
_DUR_MS = re.compile(r"(?<=: )\d+(?:\.\d+)?(ms|s)\b")
_DUR_CLOCK = re.compile(r"\b\d+:\d{2}\.\d{3} \((h:mm|m):ss\.mmm\)")
_FRAME = re.compile(r"^\s{4,}(at\s|\d+:\s)|… \d+ more identical frames")
_TEST_DURATION = re.compile(r"\(\d+(?:\.\d+)?(?:ms|s)\)")
_TEST_LOCATION = re.compile(r"^(\s*)test at .+:\d+:\d+$")


def normalize(text: str) -> str:
out = []
for line in text.split("\n"):
if _FRAME.search(line):
continue
location = _TEST_LOCATION.match(line)
if location:
line = f"{location.group(1)}test at <location>"
line = _DUR_MS.sub(lambda m: "<dur>" + m.group(1), line)
line = _DUR_CLOCK.sub(lambda m: "<dur:" + m.group(1) + ">", line)
line = _TEST_DURATION.sub("(<test-duration>)", line)
stripped = line.lstrip()
if stripped.startswith("ℹ duration_ms "):
indent = line[: len(line) - len(stripped)]
line = f"{indent}ℹ duration_ms <test-duration>"
out.append(line)
return "\n".join(out)

Expand Down
70 changes: 70 additions & 0 deletions test-parity/node-suite/test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# `node:test` granular parity coverage

This directory compares deterministic `node:test` semantics through controlled
console markers. The differential runner still compares the test runner's own
stdout and exit status, but canonicalizes test durations and source locations;
fixtures do not assert paths, stacks, process IDs, or terminal formatting.

## Upstream selection

The expansion was selected on 2026-07-16 from these primary repository
snapshots:

- Node.js [`34c28d5a69f4f00cd599adcbe57834435d3a683b`](https://github.com/nodejs/node/tree/34c28d5a69f4f00cd599adcbe57834435d3a683b), especially
[`test-runner-mocking.js`](https://github.com/nodejs/node/blob/34c28d5a69f4f00cd599adcbe57834435d3a683b/test/parallel/test-runner-mocking.js),
[`test-runner-plan.mjs`](https://github.com/nodejs/node/blob/34c28d5a69f4f00cd599adcbe57834435d3a683b/test/parallel/test-runner-plan.mjs),
[`test-runner-aftereach-runtime-skip.js`](https://github.com/nodejs/node/blob/34c28d5a69f4f00cd599adcbe57834435d3a683b/test/parallel/test-runner-aftereach-runtime-skip.js),
[`test-runner-subtest-after-hook.js`](https://github.com/nodejs/node/blob/34c28d5a69f4f00cd599adcbe57834435d3a683b/test/parallel/test-runner-subtest-after-hook.js), and the deterministic parts of the run and mock-timer tests.
- Deno [`f8a17c8171569fa2870d740030aaa59c91fdf9ee`](https://github.com/denoland/deno/tree/f8a17c8171569fa2870d740030aaa59c91fdf9ee). Deno's current Node-compat selection does not carry a dedicated `node:test` file under `tests/unit_node`; its runner, context, hooks, mocks, timer, snapshot, and reporter compatibility lives in
[`ext/node/polyfills/testing.ts`](https://github.com/denoland/deno/blob/f8a17c8171569fa2870d740030aaa59c91fdf9ee/ext/node/polyfills/testing.ts).
- Bun [`6173d6431ee8ad086bf79d1d5354080cfe937964`](https://github.com/oven-sh/bun/tree/6173d6431ee8ad086bf79d1d5354080cfe937964), especially its
[`node:test` selection](https://github.com/oven-sh/bun/blob/6173d6431ee8ad086bf79d1d5354080cfe937964/test/js/node/test_runner/node-test.test.ts) and
[hook-order fixture](https://github.com/oven-sh/bun/blob/6173d6431ee8ad086bf79d1d5354080cfe937964/test/js/node/test_runner/fixtures/02-hooks.js).

## Added diagnostic categories

- `imports` and `runner/registration` (10 fixtures): export aliases, callback
overloads, deferred registration, async/callback completion, option
validation, nested suites/subtests, and parent-child completion.
- `runner/context` and `runner/api`: assertion surface, plans, runtime
skip/todo/only, failure propagation, names, the existing diagnostic case, and
the claimed `run()` surface (16 fixtures).
- `runner/hooks` (9 fixtures): global, repeated, and nested ordering, hook
context, runtime-skip cleanup, and cleanup after body or setup throws.
- `mock-fn` (27 fixtures): successful, async, bound, no-op, and throwing call
records; receiver/prototype/inheritance behavior; implementation queues and
indexed overrides; reset/restore behavior; property/accessor contexts;
symbols, descriptors, `times`, and validation.
- `mock-timers` (13 fixtures): timeout arguments and cancellation, interval
repetition/self-clear, reset, validation, boundary ordering, nested
`runAll()`, Date construction, and deterministic `setTime()` behavior.
- `snapshots` and `reporters` (4 fixtures): serializer/assertion validation and
synthetic directive/nesting events.

Snapshot fixtures use fixed local files, and reporter fixtures feed synthetic
events. The 79 added fixtures expand the module from 11 to 90 cases without
depending on wall-clock time or external files.

## Stopping judgment

The remaining Node runner corpus is intentionally left for separate work:

- CLI discovery, watch mode, coverage, source maps, process isolation, worker
IDs, global setup, rerun state, and force-exit behavior require subprocess or
multi-file harness support rather than this in-process granular suite.
- concurrency, randomization, timeouts, abort scheduling, refed handles, and
scheduler-sensitive timer APIs are not deterministic enough for byte-for-byte
stdout comparison here.
- enabling mock timers twice currently leaves a Perry process handle alive even
after `reset()`; that validation belongs with the runtime cleanup fix rather
than a granular case that times out the harness.
- colors/TTY, absolute locations, stacks, durations, and reporter formatting
tied to those values are environment-specific.
- `TestContext.waitFor()`, `runOnly()`, tags, full names, signals, and custom
assertions are not listed in Perry's current `node:test` manifest; testing
those as claimed compatibility would overstate the supported surface.
- module mocking and snapshot update/CLI behavior are separate runtime and CLI
projects. Constructor-target mocks are also deferred because Perry does not
currently expose constructable mock wrappers. Further core cases are
redundant with the focused fixtures above or depend on one of these excluded
surfaces.
6 changes: 6 additions & 0 deletions test-parity/node-suite/test/imports/aliases.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import test, { describe, it, suite } from "node:test";

console.log("test.test identity:", test.test === test);
console.log("it identity:", it === test);
console.log("describe identity:", describe === suite);
console.log("method identities:", test.skip === test.skip, test.todo === test.todo);
18 changes: 18 additions & 0 deletions test-parity/node-suite/test/mock-fn/accessor-options.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { mock } from "node:test";

let stored = "initial";
const target = {
get value() {
return stored;
},
set value(value: string) {
stored = value;
},
};

const getter = mock.getter(target, "value", {});
const setter = mock.setter(target, "value", {});
console.log("getter options:", target.value, getter.mock.callCount());
target.value = "changed";
console.log("setter options:", stored, setter.mock.callCount());
mock.restoreAll();
24 changes: 24 additions & 0 deletions test-parity/node-suite/test/mock-fn/accessor-preservation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { mock } from "node:test";

let stored = 1;
const target = {
get value() {
return stored;
},
set value(value: number) {
stored = value;
},
};

const getter = mock.getter(target, "value", () => 10);
target.value = 4;
console.log("getter preserves setter:", target.value, stored);
getter.mock.restore();

const setter = mock.setter(target, "value", (value: number) => {
stored = value * 2;
});
console.log("setter preserves getter:", target.value);
target.value = 5;
console.log("setter result:", stored, target.value);
setter.mock.restore();
7 changes: 7 additions & 0 deletions test-parity/node-suite/test/mock-fn/async-result.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { mock } from "node:test";

const fn = mock.fn(async (value: number) => value * 2);
const promise = fn(4);
console.log("record promise:", fn.mock.calls[0].result === promise, fn.mock.callCount());
console.log("resolved:", await promise);
mock.restoreAll();
10 changes: 10 additions & 0 deletions test-parity/node-suite/test/mock-fn/bound.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { mock } from "node:test";

const fn = mock.fn(function (this: any, a: number, b: number) {
return this.offset + a + b;
});
const bound = fn.bind({ offset: 10 }, 2);
console.log("bound result:", bound(3));
console.log("bound count:", fn.mock.callCount(), JSON.stringify(fn.mock.calls[0].arguments));
console.log("bound this:", fn.mock.calls[0].this.offset);
mock.restoreAll();
33 changes: 33 additions & 0 deletions test-parity/node-suite/test/mock-fn/call-record-errors.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import { mock } from "node:test";

const expected = new Error("controlled mock failure");
const fn = mock.fn(function (this: any, value: number) {
if (value < 0) throw expected;
return this.offset + value;
});

console.log("success:", fn.call({ label: "receiver", offset: 4 }, 3));
try {
fn(-1);
} catch (error) {
console.log("caught:", error === expected);
}

const success = fn.mock.calls[0];
const failure = fn.mock.calls[1];
console.log(
"success record:",
JSON.stringify(success.arguments),
success.this.label,
success.result,
success.error === undefined,
success.target === undefined,
);
console.log(
"failure record:",
JSON.stringify(failure.arguments),
failure.result === undefined,
failure.error === expected,
typeof failure.stack,
);
mock.restoreAll();
11 changes: 11 additions & 0 deletions test-parity/node-suite/test/mock-fn/call-record-order.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { mock } from "node:test";

let inside = -1;
const fn = mock.fn(() => {
inside = fn.mock.callCount();
return "value";
});

console.log("call result:", fn());
console.log("counts:", inside, fn.mock.callCount());
mock.restoreAll();
13 changes: 13 additions & 0 deletions test-parity/node-suite/test/mock-fn/idempotent-restore.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { mock } from "node:test";

const target = {
read() {
return "original";
},
};
const method = mock.method(target, "read", () => "mocked");
console.log("mocked:", target.read());
method.mock.restore();
console.log("first restore:", target.read());
method.mock.restore();
console.log("second restore:", target.read());
13 changes: 13 additions & 0 deletions test-parity/node-suite/test/mock-fn/implementations.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { mock } from "node:test";

const fn = mock.fn((value: number) => `original:${value}`);
fn.mock.mockImplementation((value: number) => `replacement:${value}`);
fn.mock.mockImplementationOnce((value: number) => `once-a:${value}`);
fn.mock.mockImplementationOnce((value: number) => `once-b:${value}`);

console.log("sequence:", fn(1), fn(2), fn(3));
console.log("count:", fn.mock.callCount(), fn.mock.calls.length);
fn.mock.resetCalls();
console.log("reset calls:", fn.mock.callCount(), fn(4));
fn.mock.restore();
console.log("restore:", fn(5), fn.mock.callCount());
12 changes: 12 additions & 0 deletions test-parity/node-suite/test/mock-fn/inherited-method.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { mock } from "node:test";

const prototype = {
read() {
return "prototype";
},
};
const target = Object.create(prototype);
const method = mock.method(target, "read", () => "mocked");
console.log("inherited mocked:", target.read(), Object.hasOwn(target, "read"));
method.mock.restore();
console.log("inherited restored:", target.read(), Object.hasOwn(target, "read"));
19 changes: 19 additions & 0 deletions test-parity/node-suite/test/mock-fn/method-options-validation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { mock } from "node:test";

function codeOf(fn: () => void): string {
try {
fn();
return "NO_THROW";
} catch (error) {
return (error as any).code ?? (error as Error).name;
}
}

const target = { method() {} };
console.log("options number:", codeOf(() => mock.method(target, "method", () => {}, 5 as any)));
console.log(
"getter setter:",
codeOf(() => mock.method(target, "method", { getter: true, setter: true } as any)),
);
console.log("getter false:", codeOf(() => mock.getter(target, "method", { getter: false } as any)));
mock.restoreAll();
14 changes: 14 additions & 0 deletions test-parity/node-suite/test/mock-fn/method-receiver.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { mock } from "node:test";

const target = {
base: 4,
add(value: number) {
return this.base + value;
},
};

const method = mock.method(target, "add");
console.log("method result:", target.add(3));
const call = method.mock.calls[0];
console.log("method record:", call.this === target, JSON.stringify(call.arguments), call.result);
method.mock.restore();
16 changes: 16 additions & 0 deletions test-parity/node-suite/test/mock-fn/method-restore-chain.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { mock } from "node:test";

const target = {
value: 10,
read() {
return this.value;
},
};

const first = mock.method(target, "read", () => 20);
const second = mock.method(target, "read", () => 30);
console.log("nested:", target.read(), first === second);
second.mock.restore();
console.log("restore second:", target.read(), target.read === first);
first.mock.restore();
console.log("restore first:", target.read(), (target.read as any).mock === undefined);
15 changes: 15 additions & 0 deletions test-parity/node-suite/test/mock-fn/name-length.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { mock } from "node:test";

function original(a: unknown, b: unknown) {
return [a, b];
}

const fn = mock.fn(original);
console.log("name:", original.name, fn.name);
console.log("length:", original.length, fn.length);
console.log(
"descriptors:",
Object.getOwnPropertyDescriptor(fn, "name")?.configurable,
Object.getOwnPropertyDescriptor(fn, "length")?.configurable,
);
mock.restoreAll();
15 changes: 15 additions & 0 deletions test-parity/node-suite/test/mock-fn/no-op.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { mock } from "node:test";

const fn = mock.fn();
console.log("initial:", fn.mock.callCount(), fn.mock.calls.length);
console.log("result:", fn("a", 2));
const call = fn.mock.calls[0];
console.log(
"record:",
JSON.stringify(call.arguments),
call.result === undefined,
call.error === undefined,
call.this === undefined,
call.target === undefined,
);
mock.restoreAll();
16 changes: 16 additions & 0 deletions test-parity/node-suite/test/mock-fn/non-configurable.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { mock } from "node:test";

const target: any = {};
Object.defineProperty(target, "method", {
configurable: false,
value() {
return "original";
},
});

try {
mock.method(target, "method", () => "mocked");
console.log("non-configurable: NO_THROW", target.method());
} catch (error) {
console.log("non-configurable:", (error as any).code ?? (error as Error).name, target.method());
}
9 changes: 9 additions & 0 deletions test-parity/node-suite/test/mock-fn/once-indices.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { mock } from "node:test";

const fn = mock.fn(() => "default");
fn();
fn.mock.mockImplementationOnce(() => "third", 2);
fn.mock.mockImplementationOnce(() => "fifth", 4);
console.log("indexed:", fn(), fn(), fn(), fn(), fn());
console.log("count:", fn.mock.callCount());
mock.restoreAll();
15 changes: 15 additions & 0 deletions test-parity/node-suite/test/mock-fn/property-access-records.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { mock } from "node:test";

const target = { value: 1 };
const property = mock.property(target, "value", 2);
console.log("get:", target.value);
target.value = 3;
console.log("set-get:", target.value);
console.log(
"accesses:",
property.mock.accessCount(),
property.mock.accesses.map((access: any) => `${access.type}:${access.value}`).join(","),
);
property.mock.resetAccesses();
console.log("reset accesses:", property.mock.accessCount(), target.value, property.mock.accessCount());
property.mock.restore();
11 changes: 11 additions & 0 deletions test-parity/node-suite/test/mock-fn/property-context.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { mock } from "node:test";

const target = { value: 1 };
const property = mock.property(target, "value", 2);
console.log("initial:", target.value, typeof property.mock);
property.mock.mockImplementation(3);
console.log("replacement:", target.value);
property.mock.mockImplementationOnce(4);
console.log("once:", target.value, target.value);
property.mock.restore();
console.log("restored:", target.value);
Loading