Skip to content

Commit 8973217

Browse files
ericmorandericmorand
committed
Add One Double Zero as coverage provider
1 parent 22029ba commit 8973217

File tree

15 files changed

+459
-40
lines changed

15 files changed

+459
-40
lines changed

‎docs/CLI.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ The directory where Jest should output its coverage files.
170170

171171
### `--coverageProvider=<provider>`
172172

173-
Indicates which provider should be used to instrument code for coverage. Allowed values are `babel` (default) or `v8`.
173+
Indicates which provider should be used to instrument code for coverage. Allowed values are `babel` (default), `v8` or `odz`.
174174

175175
### `--debug`
176176

‎docs/Configuration.md

+27-3
Original file line numberDiff line numberDiff line change
@@ -224,11 +224,35 @@ Default: `false`
224224

225225
Indicates whether the coverage information should be collected while executing the test. Because this retrofits all executed files with coverage collection statements, it may significantly slow down your tests.
226226

227-
Jest ships with two coverage providers: `babel` (default) and `v8`. See the [`coverageProvider`](#coverageprovider-string) option for more details.
227+
Jest ships with three coverage providers: `babel` (default), `v8` and `odz`. See the [`coverageProvider`](#coverageprovider-string) option for more details.
228228

229229
:::info
230230

231-
The `babel` and `v8` coverage providers use `/* istanbul ignore next */` and `/* c8 ignore next */` comments to exclude lines from coverage reports, respectively. For more information, you can view the [`istanbuljs` documentation](https://github.com/istanbuljs/nyc#parsing-hints-ignoring-lines) and the [`c8` documentation](https://github.com/bcoe/c8#ignoring-uncovered-lines-functions-and-blocks).
231+
The `babel` and `v8` coverage providers use `/* istanbul ignore next */` and `/* c8 ignore next */` comments to exclude lines from coverage reports, respectively. For more information, you can view the [`istanbuljs` documentation](https://github.com/istanbuljs/nyc#parsing-hints-ignoring-lines) and the [`c8` documentation](https://github.com/bcoe/c8#ignoring-uncovered-lines-functions-and-blocks). The `odz` coverage provider doesn't support exclusion comment.
232+
233+
The `v8` coverage provider comes with the following tradeoffs:
234+
235+
- It is not a 1:1 replacement for Babel/Istanbul coverage
236+
237+
- Switching between the two will usually change the reported coverage statistics, which can change whether coverage thresholds are reached or not
238+
- Switching between the two can cause regions of uncovered code to be discovered or ignored (This is mostly just an overall summary of the other points)
239+
240+
- It works by taking a coverage report for output/transpiled code, and then using `v8-to-istanbul` and source maps to convert that report into an Istanbul-compatible format
241+
- This is an inherently imprecise and heuristic-driven process, though in many cases it works well enough for practical purposes
242+
- In some cases this can give confusing or misleading results, or fail to distinguish between user code and generated code (e.g. uncovered branches introduced by `__esModule` detection shims)
243+
- Babel/Istanbul is able to be more precise because it usually operates directly on the user's original source code
244+
- It tracks “blocks”, not individual statements
245+
- In particular, if you have a sequence of statements, and the middle statements always throw an exception, V8 coverage will mark the later statements as “covered” even though they never ran
246+
- This is a deliberate tradeoff made by the V8 developers who implemented coverage
247+
- Babel/Istanbul is able to be more precise because it explicitly instruments every source statement
248+
- It does not track the else branch of an if-statement without an explicit else
249+
250+
- So if a one-sided if-statement's condition is always true, V8 will not warn about an uncovered branch
251+
- Babel/Istanbul is able to track these by artificially inserting an else with a branch counter
252+
253+
- It does not respect the `collectCoverageFrom` Jest configuration: regardless of the value of `collectCoverageFrom`, it emits coverage report for whatever file that was encountered during the execution of the tests.
254+
255+
The `odz` coverage provider also makes use of V8 coverage data, but doesn't come any of the `v8` provider tradeoffs because it operates at the AST level.
232256

233257
:::
234258

@@ -314,7 +338,7 @@ These pattern strings match against the full path. Use the `<rootDir>` string to
314338

315339
### `coverageProvider` \[string]
316340

317-
Indicates which provider should be used to instrument code for coverage. Allowed values are `babel` (default) or `v8`.
341+
Indicates which provider should be used to instrument code for coverage. Allowed values are `babel` (default), `v8` and `odz`.
318342

319343
### `coverageReporters` \[array&lt;string | \[string, options]&gt;]
320344

‎e2e/__tests__/__snapshots__/coverageProviderODZ.test.ts.snap

+116
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
// Jest Snapshot v1, https://goo.gl/fbAQLP
2+
3+
exports[`prints correct coverage report, if a CJS module is put under test without transformation 1`] = `
4+
" console.log
5+
this will print
6+
7+
at covered (module.js:11:11)
8+
9+
--------------|---------|----------|---------|---------|-------------------
10+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
11+
--------------|---------|----------|---------|---------|-------------------
12+
All files | 60 | 50 | 50 | 60 |
13+
module.js | 66.66 | 50 | 50 | 66.66 | 14-15,19
14+
uncovered.js | 0 | 100 | 100 | 0 | 8
15+
--------------|---------|----------|---------|---------|-------------------"
16+
`;
17+
18+
exports[`prints correct coverage report, if a TS module is transpiled by Babel to CJS and put under test 1`] = `
19+
" console.log
20+
this will print
21+
22+
at log (module.ts:13:11)
23+
24+
--------------|---------|----------|---------|---------|-------------------
25+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
26+
--------------|---------|----------|---------|---------|-------------------
27+
All files | 62.5 | 50 | 50 | 62.5 |
28+
module.ts | 62.5 | 50 | 50 | 62.5 | 16-17,21
29+
types.ts | 0 | 0 | 0 | 0 |
30+
uncovered.ts | 0 | 0 | 0 | 0 |
31+
--------------|---------|----------|---------|---------|-------------------"
32+
`;
33+
34+
exports[`prints correct coverage report, if a TS module is transpiled by custom transformer to ESM put under test 1`] = `
35+
" console.log
36+
this will print
37+
38+
at covered (module.ts:13:11)
39+
40+
--------------|---------|----------|---------|---------|-------------------
41+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
42+
--------------|---------|----------|---------|---------|-------------------
43+
All files | 62.5 | 50 | 50 | 62.5 |
44+
module.ts | 62.5 | 50 | 50 | 62.5 | 16-17,21
45+
types.ts | 0 | 0 | 0 | 0 |
46+
uncovered.ts | 0 | 0 | 0 | 0 |
47+
--------------|---------|----------|---------|---------|-------------------"
48+
`;
49+
50+
exports[`prints correct coverage report, if an ESM module is put under test without transformation 1`] = `
51+
" console.log
52+
this will print
53+
54+
at covered (module.js:11:11)
55+
56+
--------------|---------|----------|---------|---------|-------------------
57+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
58+
--------------|---------|----------|---------|---------|-------------------
59+
All files | 62.5 | 50 | 50 | 62.5 |
60+
module.js | 62.5 | 50 | 50 | 62.5 | 14-15,19
61+
uncovered.js | 0 | 0 | 0 | 0 |
62+
--------------|---------|----------|---------|---------|-------------------"
63+
`;
64+
65+
exports[`prints coverage with empty sourcemaps 1`] = `
66+
"----------|---------|----------|---------|---------|-------------------
67+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
68+
----------|---------|----------|---------|---------|-------------------
69+
All files | 0 | 0 | 0 | 0 |
70+
types.ts | 0 | 0 | 0 | 0 |
71+
----------|---------|----------|---------|---------|-------------------"
72+
`;
73+
74+
exports[`prints coverage with missing sourcemaps 1`] = `
75+
" console.log
76+
42
77+
78+
at Object.log (__tests__/Thing.test.js:10:9)
79+
80+
----------|---------|----------|---------|---------|-------------------
81+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
82+
----------|---------|----------|---------|---------|-------------------
83+
All files | 100 | 100 | 100 | 100 |
84+
Thing.js | 100 | 100 | 100 | 100 |
85+
x.css | 0 | 0 | 0 | 0 |
86+
----------|---------|----------|---------|---------|-------------------"
87+
`;
88+
89+
exports[`reports coverage with \`resetModules\` 1`] = `
90+
" console.log
91+
this will print
92+
93+
at log (module.js:11:11)
94+
95+
console.log
96+
this will print
97+
98+
at log (module.js:11:11)
99+
100+
--------------|---------|----------|---------|---------|-------------------
101+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
102+
--------------|---------|----------|---------|---------|-------------------
103+
All files | 60 | 50 | 50 | 60 |
104+
module.js | 66.66 | 50 | 50 | 66.66 | 14-15,19
105+
uncovered.js | 0 | 100 | 100 | 0 | 8
106+
--------------|---------|----------|---------|---------|-------------------"
107+
`;
108+
109+
exports[`vm script coverage generator 1`] = `
110+
"-------------|---------|----------|---------|---------|-------------------
111+
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
112+
-------------|---------|----------|---------|---------|-------------------
113+
All files | 80 | 75 | 66.66 | 80 |
114+
vmscript.js | 80 | 75 | 66.66 | 80 | 20-21
115+
-------------|---------|----------|---------|---------|-------------------"
116+
`;

‎e2e/__tests__/coverageProviderODZ.test.ts

+120
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
/**
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
*
4+
* This source code is licensed under the MIT license found in the
5+
* LICENSE file in the root directory of this source tree.
6+
*/
7+
8+
import * as path from 'path';
9+
import runJest from '../runJest';
10+
11+
const DIR = path.resolve(__dirname, '../coverage-provider-v8');
12+
13+
test('prints coverage with missing sourcemaps', () => {
14+
const sourcemapDir = path.join(DIR, 'no-sourcemap');
15+
16+
const {stdout, exitCode} = runJest(
17+
sourcemapDir,
18+
['--coverage', '--coverage-provider', 'odz'],
19+
{stripAnsi: true},
20+
);
21+
22+
expect(exitCode).toBe(0);
23+
expect(stdout).toMatchSnapshot();
24+
});
25+
26+
test('prints coverage with empty sourcemaps', () => {
27+
const sourcemapDir = path.join(DIR, 'empty-sourcemap');
28+
29+
const {stdout, exitCode} = runJest(
30+
sourcemapDir,
31+
['--coverage', '--coverage-provider', 'odz'],
32+
{stripAnsi: true},
33+
);
34+
35+
expect(exitCode).toBe(0);
36+
expect(stdout).toMatchSnapshot();
37+
});
38+
39+
test('reports coverage with `resetModules`', () => {
40+
const sourcemapDir = path.join(DIR, 'with-resetModules');
41+
42+
const {stdout, exitCode} = runJest(
43+
sourcemapDir,
44+
['--coverage', '--coverage-provider', 'odz'],
45+
{stripAnsi: true},
46+
);
47+
48+
expect(exitCode).toBe(0);
49+
expect(stdout).toMatchSnapshot();
50+
});
51+
52+
test('prints correct coverage report, if a CJS module is put under test without transformation', () => {
53+
const sourcemapDir = path.join(DIR, 'cjs-native-without-sourcemap');
54+
55+
const {stdout, exitCode} = runJest(
56+
sourcemapDir,
57+
['--coverage', '--coverage-provider', 'odz', '--no-cache'],
58+
{stripAnsi: true},
59+
);
60+
61+
expect(exitCode).toBe(0);
62+
expect(stdout).toMatchSnapshot();
63+
});
64+
65+
test('prints correct coverage report, if a TS module is transpiled by Babel to CJS and put under test', () => {
66+
const sourcemapDir = path.join(DIR, 'cjs-with-babel-transformer');
67+
68+
const {stdout, exitCode} = runJest(
69+
sourcemapDir,
70+
['--coverage', '--coverage-provider', 'odz', '--no-cache'],
71+
{stripAnsi: true},
72+
);
73+
74+
expect(exitCode).toBe(0);
75+
expect(stdout).toMatchSnapshot();
76+
});
77+
78+
test('prints correct coverage report, if an ESM module is put under test without transformation', () => {
79+
const sourcemapDir = path.join(DIR, 'esm-native-without-sourcemap');
80+
81+
const {stdout, exitCode} = runJest(
82+
sourcemapDir,
83+
['--coverage', '--coverage-provider', 'odz', '--no-cache'],
84+
{
85+
nodeOptions: '--experimental-vm-modules --no-warnings',
86+
stripAnsi: true,
87+
},
88+
);
89+
90+
expect(exitCode).toBe(0);
91+
expect(stdout).toMatchSnapshot();
92+
});
93+
94+
test('prints correct coverage report, if a TS module is transpiled by custom transformer to ESM put under test', () => {
95+
const sourcemapDir = path.join(DIR, 'esm-with-custom-transformer');
96+
97+
const {stdout, exitCode} = runJest(
98+
sourcemapDir,
99+
['--coverage', '--coverage-provider', 'odz', '--no-cache'],
100+
{
101+
nodeOptions: '--experimental-vm-modules --no-warnings',
102+
stripAnsi: true,
103+
},
104+
);
105+
106+
expect(exitCode).toBe(0);
107+
expect(stdout).toMatchSnapshot();
108+
});
109+
110+
test('vm script coverage generator', () => {
111+
const dir = path.resolve(__dirname, '../vmscript-coverage');
112+
const {stdout, exitCode} = runJest(
113+
dir,
114+
['--coverage', '--coverage-provider', 'odz'],
115+
{stripAnsi: true},
116+
);
117+
118+
expect(exitCode).toBe(0);
119+
expect(stdout).toMatchSnapshot();
120+
});

‎e2e/coverage-provider-v8/empty-sourcemap/package.json

+4-1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22
"name": "empty-sourcemap",
33
"version": "1.0.0",
44
"jest": {
5-
"testEnvironment": "node"
5+
"testEnvironment": "node",
6+
"collectCoverageFrom": [
7+
"<rootDir>/types.ts"
8+
]
69
}
710
}

‎e2e/coverage-provider-v8/no-sourcemap/package.json

+9-1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,14 @@
66
"transform": {
77
"\\.[jt]sx?$": "babel-jest",
88
"\\.css$": "<rootDir>/cssTransform.js"
9-
}
9+
},
10+
"moduleFileExtensions": [
11+
"js",
12+
"css"
13+
],
14+
"collectCoverageFrom": [
15+
"<rootDir>/Thing.js",
16+
"<rootDir>/x.css"
17+
]
1018
}
1119
}

‎package.json

+2-1
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,8 @@
202202
"jest": "workspace:*",
203203
"jest-environment-node": "workspace:*",
204204
"psl": "patch:psl@npm:^1.9.0#./.yarn/patches/psl-npm-1.9.0-a546edad1a.patch",
205-
"ts-node@^10.5.0": "patch:ts-node@npm:^10.5.0#./.yarn/patches/ts-node-npm-10.9.1-6c268be7f4.patch"
205+
"ts-node@^10.5.0": "patch:ts-node@npm:^10.5.0#./.yarn/patches/ts-node-npm-10.9.1-6c268be7f4.patch",
206+
"typescript": "~5.5.4"
206207
},
207208
"packageManager": "[email protected]"
208209
}

‎packages/jest-cli/src/args.ts

+3-2
Original file line numberDiff line numberDiff line change
@@ -213,8 +213,9 @@ export const options: {[key: string]: Options} = {
213213
type: 'array',
214214
},
215215
coverageProvider: {
216-
choices: ['babel', 'v8'],
217-
description: 'Select between Babel and V8 to collect coverage',
216+
choices: ['babel', 'v8', 'odz'],
217+
description:
218+
'Select between Babel, V8 and One Double Zero to collect coverage',
218219
requiresArg: true,
219220
},
220221
coverageReporters: {

‎packages/jest-reporters/package.json

+1
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
"jest-message-util": "workspace:*",
3535
"jest-util": "workspace:*",
3636
"jest-worker": "workspace:*",
37+
"one-double-zero": "1.0.0-beta.14",
3738
"slash": "^3.0.0",
3839
"string-length": "^4.0.1",
3940
"strip-ansi": "^6.0.0",

0 commit comments

Comments
 (0)