Skip to content

Commit

Permalink
test_runner,cli: mark test isolation as stable
Browse files Browse the repository at this point in the history
This commit stabilizes test isolation configuration in the
test runner.
  • Loading branch information
cjihrig committed Dec 17, 2024
1 parent 7302b6f commit 2619f38
Show file tree
Hide file tree
Showing 14 changed files with 50 additions and 48 deletions.
34 changes: 18 additions & 16 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1025,20 +1025,6 @@ generated as part of the test runner output. If no tests are run, a coverage
report is not generated. See the documentation on
[collecting code coverage from tests][] for more details.

### `--experimental-test-isolation=mode`

<!-- YAML
added: v22.8.0
-->

> Stability: 1.0 - Early development
Configures the type of test isolation used in the test runner. When `mode` is
`'process'`, each test file is run in a separate child process. When `mode` is
`'none'`, all test files run in the same process as the test runner. The default
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
present. See the [test runner execution model][] section for more information.

### `--experimental-test-module-mocks`

<!-- YAML
Expand Down Expand Up @@ -2238,8 +2224,8 @@ added:
-->

The maximum number of test files that the test runner CLI will execute
concurrently. If `--experimental-test-isolation` is set to `'none'`, this flag
is ignored and concurrency is one. Otherwise, concurrency defaults to
concurrently. If `--test-isolation` is set to `'none'`, this flag is ignored and
concurrency is one. Otherwise, concurrency defaults to
`os.availableParallelism() - 1`.

### `--test-coverage-branches=threshold`
Expand Down Expand Up @@ -2323,6 +2309,22 @@ added:
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.

### `--test-isolation=mode`

<!-- YAML
added: v22.8.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/xxxxx

Check warning on line 2318 in doc/api/cli.md

View workflow job for this annotation

GitHub Actions / lint-pr-url

pr-url doesn't match the URL of the current PR.
description: Test isolation configuration is now stable.
-->

Configures the type of test isolation used in the test runner. When `mode` is
`'process'`, each test file is run in a separate child process. When `mode` is
`'none'`, all test files run in the same process as the test runner. The default
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
present. See the [test runner execution model][] section for more information.

### `--test-name-pattern`

<!-- YAML
Expand Down
6 changes: 3 additions & 3 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,6 @@ Use this flag to enable ShadowRealm support.
.It Fl -experimental-test-coverage
Enable code coverage in the test runner.
.
.It Fl -experimental-test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
.It Fl -experimental-test-module-mocks
Enable module mocking in the test runner.
.
Expand Down Expand Up @@ -455,6 +452,9 @@ Require a minimum threshold for line coverage (0 - 100).
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.
.
.It Fl -test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
.It Fl -test-name-pattern
A regular expression that configures the test runner to only execute tests
whose name matches the provided pattern.
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/test_runner/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ function parseCommandLine() {
}

if (isTestRunner) {
isolation = getOptionValue('--experimental-test-isolation');
isolation = getOptionValue('--test-isolation');
timeout = getOptionValue('--test-timeout') || Infinity;

if (isolation === 'none') {
Expand Down
8 changes: 4 additions & 4 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ void EnvironmentOptions::CheckOptions(std::vector<std::string>* errors,
debug_options_.allow_attaching_debugger = true;
} else {
if (test_isolation != "process") {
errors->push_back("invalid value for --experimental-test-isolation");
errors->push_back("invalid value for --test-isolation");
}

#ifndef ALLOW_ATTACHING_DEBUGGER_IN_TEST_RUNNER
Expand Down Expand Up @@ -684,10 +684,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
"the line coverage minimum threshold",
&EnvironmentOptions::test_coverage_lines,
kAllowedInEnvvar);

AddOption("--experimental-test-isolation",
AddOption("--test-isolation",
"configures the type of test isolation used in the test runner",
&EnvironmentOptions::test_isolation);
&EnvironmentOptions::test_isolation,
kAllowedInEnvvar);
AddOption("--experimental-test-module-mocks",
"enable module mocking in the test runner",
&EnvironmentOptions::test_runner_module_mocks);
Expand Down
4 changes: 2 additions & 2 deletions test/parallel/test-runner-cli-concurrency.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ test('concurrency of two', async () => {
});

test('isolation=none uses a concurrency of one', async () => {
const args = ['--test', '--experimental-test-isolation=none'];
const args = ['--test', '--test-isolation=none'];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /concurrency: 1,/);
});

test('isolation=none overrides --test-concurrency', async () => {
const args = [
'--test', '--experimental-test-isolation=none', '--test-concurrency=2',
'--test', '--test-isolation=none', '--test-concurrency=2',
];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /concurrency: 1,/);
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-cli-timeout.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ test('timeout of 10ms', async () => {

test('isolation=none uses the --test-timeout flag', async () => {
const args = [
'--test', '--experimental-test-isolation=none', '--test-timeout=10',
'--test', '--test-isolation=none', '--test-timeout=10',
];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /timeout: 10,/);
Expand Down
18 changes: 9 additions & 9 deletions test/parallel/test-runner-cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ for (const isolation of ['none', 'process']) {
// File not found.
const args = [
'--test',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
'a-random-file-that-does-not-exist.js',
];
const child = spawnSync(process.execPath, args);
Expand All @@ -27,7 +27,7 @@ for (const isolation of ['none', 'process']) {
// Default behavior. node_modules is ignored. Files that don't match the
// pattern are ignored except in test/ directories.
const args = ['--test', '--test-reporter=tap',
`--experimental-test-isolation=${isolation}`];
`--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'default-behavior') });

assert.strictEqual(child.status, 1);
Expand All @@ -46,7 +46,7 @@ for (const isolation of ['none', 'process']) {
{
// Should match files with "-test.(c|m)js" suffix.
const args = ['--test', '--test-reporter=tap',
`--experimental-test-isolation=${isolation}`];
`--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'matching-patterns') });

assert.strictEqual(child.status, 0);
Expand All @@ -64,7 +64,7 @@ for (const isolation of ['none', 'process']) {
for (const type of ['strip', 'transform']) {
// Should match files with "-test.(c|m)(t|j)s" suffix when typescript support is enabled
const args = ['--test', '--test-reporter=tap', '--no-warnings',
`--experimental-${type}-types`, `--experimental-test-isolation=${isolation}`];
`--experimental-${type}-types`, `--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'matching-patterns') });

if (!process.config.variables.node_use_amaro) {
Expand All @@ -91,7 +91,7 @@ for (const isolation of ['none', 'process']) {
'--require', join(testFixtures, 'protoMutation.js'),
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'default-behavior') });

Expand All @@ -112,7 +112,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
join(testFixtures, 'index.js'),
];
const child = spawnSync(process.execPath, args, { cwd: testFixtures });
Expand All @@ -129,7 +129,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
join(testFixtures, 'default-behavior/node_modules/*.js'),
];
const child = spawnSync(process.execPath, args);
Expand All @@ -143,7 +143,7 @@ for (const isolation of ['none', 'process']) {

{
// The current directory is used by default.
const args = ['--test', `--experimental-test-isolation=${isolation}`];
const args = ['--test', `--test-isolation=${isolation}`];
const options = { cwd: join(testFixtures, 'default-behavior') };
const child = spawnSync(process.execPath, args, options);

Expand All @@ -165,7 +165,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
'test/fixtures/test-runner/default-behavior/index.test.js',
'test/fixtures/test-runner/nested.js',
'test/fixtures/test-runner/invalid-tap.js',
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-coverage.js
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ test.skip('coverage works with isolation=none', skipIfNoInspector, () => {
'--experimental-test-coverage',
'--test-reporter',
'tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
];
const result = spawnSync(process.execPath, args, {
env: { ...process.env, NODE_TEST_TMPDIR: tmpdir.path },
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-extraneous-async-activity.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ const { spawnSync } = require('child_process');
{
const child = spawnSync(process.execPath, [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixtures.path('test-runner', 'async-error-in-test-hook.mjs'),
]);
const stdout = child.stdout.toString();
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-force-exit-failure.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ for (const isolation of ['none', 'process']) {
'--test',
'--test-reporter=spec',
'--test-force-exit',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
fixture,
];
const r = spawnSync(process.execPath, args);
Expand Down
8 changes: 4 additions & 4 deletions test/parallel/test-runner-no-isolation-filtering.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test('works with --test-only', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-only',
fixture1,
fixture2,
Expand All @@ -35,7 +35,7 @@ test('works without --test-only', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture1,
fixture2,
];
Expand All @@ -57,7 +57,7 @@ test('works with --test-name-pattern', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-name-pattern=/test one/',
fixture1,
fixture2,
Expand All @@ -75,7 +75,7 @@ test('works with --test-skip-pattern', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-skip-pattern=/one/',
fixture1,
fixture2,
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-no-isolation-hooks.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { test } from 'node:test';

const testArguments = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
];

const testFiles = [
Expand Down
6 changes: 3 additions & 3 deletions test/parallel/test-runner-snapshot-tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('fails prior to snapshot generation', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture,
fixture2,
];
Expand All @@ -370,7 +370,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('passes when regenerating snapshots', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-update-snapshots',
fixture,
fixture2,
Expand All @@ -391,7 +391,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('passes when snapshots exist', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture,
fixture2,
];
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-watch-mode.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ async function testWatch({
const ran2 = Promise.withResolvers();
const child = spawn(process.execPath,
['--watch', '--test', '--test-reporter=spec',
isolation ? `--experimental-test-isolation=${isolation}` : '',
isolation ? `--test-isolation=${isolation}` : '',
file ? fixturePaths[file] : undefined].filter(Boolean),
{ encoding: 'utf8', stdio: 'pipe', cwd: tmpdir.path });
let stdout = '';
Expand Down

0 comments on commit 2619f38

Please sign in to comment.