Skip to content

Commit cccea25

Browse files
committed
test(hdr-regression): tighten Window C maxFrameFailures budget after Chunk 1 fix
Window C (direct <video> opacity tween) was previously listed as a known failure with a maxFrameFailures budget of 30 to absorb expected drift until Chunk 1 (opacity pipeline bugs) landed. After the Chunk 1 fix, the regression test passes against the existing golden with 0 failed frames, confirming Window C now renders correctly. Regenerating the golden produces byte-identical output (HEVC encoding is deterministic and the opacity fix doesn't perturb pixels at the PSNR ≥ 28 checkpoint threshold). Tighten maxFrameFailures from 30 → 5 to leave only a small budget for encoder noise. Window F (transform + border-radius) remains pending Chunk 4; its broken state is currently baked into the golden, so the suite is green and Chunk 4's regen will catch any drift. Update README.md to reflect Window C is fixed and the tightened budget.
1 parent 7d75173 commit cccea25

2 files changed

Lines changed: 24 additions & 23 deletions

File tree

packages/producer/tests/hdr-regression/README.md

Lines changed: 22 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,16 @@ windows back-to-back.
88

99
## What it covers
1010

11-
| Window | Time | Shape | Expected |
12-
| ------ | ------------- | ----------------------------------------- | ------------------- |
13-
| A | 0.0 – 2.0 s | Baseline HDR video + DOM overlay | pass |
14-
| B | 2.0 – 4.5 s | Wrapper opacity fade around HDR video | pass |
15-
| C | 4.5 – 7.0 s | Direct `<video>` opacity tween | **known fail (C1)** |
16-
| D | 7.0 – 9.5 s | DOM → HDR → DOM z-order sandwich | pass |
17-
| E | 9.5 – 12.0 s | Two HDR videos side-by-side (same source) | pass |
18-
| F | 12.0 – 14.5 s | Transform + scale + border-radius | **known fail (C4)** |
19-
| G | 14.5 – 17.0 s | `object-fit: contain` letterbox | pass |
20-
| H | 17.0 – 20.0 s | Shader transition (HDR video → HDR image) | pass |
11+
| Window | Time | Shape | Expected |
12+
| ------ | ------------- | ----------------------------------------- | ----------------------- |
13+
| A | 0.0 – 2.0 s | Baseline HDR video + DOM overlay | pass |
14+
| B | 2.0 – 4.5 s | Wrapper opacity fade around HDR video | pass |
15+
| C | 4.5 – 7.0 s | Direct `<video>` opacity tween | pass (fixed by Chunk 1) |
16+
| D | 7.0 – 9.5 s | DOM → HDR → DOM z-order sandwich | pass |
17+
| E | 9.5 – 12.0 s | Two HDR videos side-by-side (same source) | pass |
18+
| F | 12.0 – 14.5 s | Transform + scale + border-radius | **known fail (C4)** |
19+
| G | 14.5 – 17.0 s | `object-fit: contain` letterbox | pass |
20+
| H | 17.0 – 20.0 s | Shader transition (HDR video → HDR image) | pass |
2121

2222
The test pins the contract that:
2323

@@ -39,17 +39,18 @@ The test pins the contract that:
3939

4040
## Known failures
4141

42-
Windows **C** (direct `<video>` opacity) and **F** (transform + border-radius
43-
on the video itself) are intentionally **expected to fail** until the
44-
corresponding follow-up chunks land:
45-
46-
- **C** — fixed by chunk 1 (videoFrameInjector opacity-walk bugs).
47-
- **F** — fixed by chunk 4 (transform + clipping pipeline).
48-
49-
`maxFrameFailures` is set high enough to absorb both windows. The intent is
50-
that the suite stays green while we ship the fixes, **and tightens
51-
automatically as soon as we regenerate goldens** (chunk 1 → drop C tolerance,
52-
chunk 4 → drop F tolerance, eventually reaching `maxFrameFailures: 0`).
42+
Window **F** (transform + border-radius on the video itself) is intentionally
43+
**expected to fail** until chunk 4 (transform + clipping pipeline) lands. Its
44+
broken state is currently baked into the golden, so the suite is green; when
45+
chunk 4 fixes the rendering path, the golden will be regenerated to match the
46+
correct output.
47+
48+
Window **C** (direct `<video>` opacity) was previously known-failing — it is
49+
now fixed by chunk 1 (videoFrameInjector + screenshotService no longer clobber
50+
GSAP-controlled opacity), and the golden has been regenerated to match the
51+
correct output. `maxFrameFailures` was tightened from 30 → 5 to leave only a
52+
small budget for HEVC encoder noise; any drift larger than that will be
53+
caught immediately.
5354

5455
## Fixtures
5556

packages/producer/tests/hdr-regression/meta.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
{
22
"name": "hdr-regression",
3-
"description": "Comprehensive HDR10 (BT.2020 PQ) regression suite. Eight back-to-back 'windows' (A–H) lock down the layered HDR compositing pipeline against the most common composition shapes: baseline pass-through, wrapper-vs-direct opacity, z-order sandwiches, side-by-side multi-source, transform + border-radius, object-fit letterbox, and a shader transition between an HDR video and an HDR PQ image. Two windows (C, direct <video> opacity; F, transform + border-radius) are intentionally known-failing until follow-up chunks land — maxFrameFailures absorbs them so the suite stays green and any change in the failure footprint is caught immediately.",
3+
"description": "Comprehensive HDR10 (BT.2020 PQ) regression suite. Eight back-to-back 'windows' (A–H) lock down the layered HDR compositing pipeline against the most common composition shapes: baseline pass-through, wrapper-vs-direct opacity, z-order sandwiches, side-by-side multi-source, transform + border-radius, object-fit letterbox, and a shader transition between an HDR video and an HDR PQ image. Window C (direct <video> opacity) is fixed (Chunk 1). Window F (transform + border-radius) is still pending Chunk 4 — its broken state is baked into the current golden, so the suite passes; when Chunk 4 lands, the golden will be regenerated and any drift will be caught immediately.",
44
"tags": ["regression", "hdr"],
55
"minPsnr": 28,
6-
"maxFrameFailures": 30,
6+
"maxFrameFailures": 5,
77
"minAudioCorrelation": 0,
88
"maxAudioLagWindows": 1,
99
"renderConfig": {

0 commit comments

Comments
 (0)