Skip to content

Commit 48de9ed

Browse files
iuioiuabartlomieju
authored andcommitted
docs: move WPT guide to tests/wpt (#28695)
Part of denoland/docs#1602 An argument could be made that this documentation should instead be under a WPT section within `tests/README.md`. Closes #27081
1 parent af0ff09 commit 48de9ed

File tree

1 file changed

+139
-0
lines changed

1 file changed

+139
-0
lines changed

tests/wpt/README.md

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Web Platform Test (WPT)
2+
3+
Deno uses a custom test runner for Web Platform Tests. It can be found at
4+
`./tests/wpt/wpt.ts`, relative to the root of this codebase.
5+
6+
## Running tests
7+
8+
> If you are on Windows, or your system does not support hashbangs, prefix all
9+
> `./tests/wpt/wpt.ts` commands with
10+
> `deno run --unstable --allow-write --allow-read --allow-net --allow-env --allow-run`.
11+
12+
Before attempting to run WPT tests for the first time, please run the WPT setup.
13+
You must also run this command every time the `./test_util/wpt` submodule is
14+
updated:
15+
16+
```shell
17+
./tests/wpt/wpt.ts setup
18+
```
19+
20+
To run all available web platform tests, run the following command:
21+
22+
```shell
23+
./tests/wpt/wpt.ts run
24+
25+
# You can also filter which test files to run by specifying filters:
26+
./tests/wpt/wpt.ts run -- streams/piping/general hr-time
27+
```
28+
29+
The test runner will run each web platform test and record its status (failed or
30+
ok). It will then compare this output to the expected output of each test as
31+
specified in the `./tests/wpt/runner/expectation.json` file. This file is a
32+
nested JSON structure that mirrors the `./tests/wpt/suite` directory. It
33+
describes for each test file, if it should pass as a whole (all tests pass,
34+
`true`), if it should fail as a whole (test runner encounters an exception
35+
outside of a test or all tests fail, `false`), or which tests it expects to fail
36+
(a string array of test case names).
37+
38+
## Updating enabled tests or expectations
39+
40+
You can update the `./tests/wpt/runner/expectation.json` file manually by
41+
changing the value of each of the test file entries in the JSON structure. The
42+
alternative and preferred option is to have the WPT runner run all, or a
43+
filtered subset of tests, and then automatically update the `expectation.json`
44+
file to match the current reality. You can do this with the `./wpt.ts update`
45+
command. Example:
46+
47+
```shell
48+
./tests/wpt/wpt.ts update -- hr-time
49+
```
50+
51+
After running this command the `expectation.json` file will match the current
52+
output of all the tests that were run. This means that running `wpt.ts run`
53+
right after a `wpt.ts update` should always pass.
54+
55+
## Subcommands
56+
57+
### `setup`
58+
59+
Validate that your environment is configured correctly, or help you configure
60+
it.
61+
62+
This will check that the python3 (or `python.exe` on Windows) is actually
63+
Python 3.
64+
65+
You can specify the following flags to customize behaviour:
66+
67+
```console
68+
--rebuild
69+
Rebuild the manifest instead of downloading. This can take up to 3 minutes.
70+
71+
--auto-config
72+
Automatically configure /etc/hosts if it is not configured (no prompt will be shown).
73+
```
74+
75+
### `run`
76+
77+
Run all tests like specified in `expectation.json`.
78+
79+
You can specify the following flags to customize behaviour:
80+
81+
```console
82+
--release
83+
Use the ./target/release/deno binary instead of ./target/debug/deno
84+
85+
--quiet
86+
Disable printing of `ok` test cases.
87+
88+
--json=<file>
89+
Output the test results as JSON to the file specified.
90+
```
91+
92+
You can also specify exactly which tests to run by specifying one of more
93+
filters after a `--`:
94+
95+
```console
96+
./tests/wpt/wpt.ts run -- hr-time streams/piping/general
97+
```
98+
99+
### `update`
100+
101+
Update the `expectation.json` to match the current reality.
102+
103+
You can specify the following flags to customize behaviour:
104+
105+
```console
106+
--release
107+
Use the ./target/release/deno binary instead of ./target/debug/deno
108+
109+
--quiet
110+
Disable printing of `ok` test cases.
111+
112+
--json=<file>
113+
Output the test results as JSON to the file specified.
114+
```
115+
116+
You can also specify exactly which tests to run by specifying one of more
117+
filters after a `--`:
118+
119+
```console
120+
./tests/wpt/wpt.ts update -- hr-time streams/piping/general
121+
```
122+
123+
## FAQ
124+
125+
### Upgrading the wpt submodule:
126+
127+
```shell
128+
cd tests/wpt/suite
129+
git fetch origin
130+
git checkout origin/epochs/daily
131+
cd ../../../
132+
git add ./tests/wpt/suite
133+
```
134+
135+
All contributors will need to rerun `./tests/wpt/wpt.ts setup` after this.
136+
137+
Since upgrading WPT usually requires updating the expectations to cover all
138+
sorts of upstream changes, it's best to do that as a separate PR, rather than as
139+
part of a PR that implements a fix or feature.

0 commit comments

Comments
 (0)