Skip to content

chore(expect): pass expect title explicitly to library calls - #42250

Merged
Dmitry Gozman (dgozman) merged 2 commits into
microsoft:mainfrom
dgozman:expect-explicit-title
Aug 14, 2026
Merged

Dmitry Gozman (dgozman) merged 2 commits into
microsoft:mainfrom
dgozman:expect-explicit-title

Conversation

@dgozman

Copy link
Copy Markdown
Collaborator

Summary

  • Matchers pass their step title explicitly to locator._expect, frame._expect and page._expectScreenshot, instead of the test runner plumbing apiName/title into the api call through the instrumentation listener.
  • The apiName is inferred from the stack, falling back to the explicit title for internal methods like locator._expect.
  • The test runner instrumentation matches api calls by channel type/method, and apiName is removed from ApiCallData.

Matchers now pass their step title to locator._expect and friends,
instead of the test runner plumbing apiName/title into the api call
through the instrumentation listener. The apiName is inferred from the
stack, falling back to the explicit title for internal methods like
locator._expect. The test runner instrumentation matches api calls by
channel type/method instead of the apiName.
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a look at the CI failures here.

🔴 One failure looks caused by this PR

playwright-packages-install-behavior.spec.ts:99 › @playwright/test should work fails on all three OSes, and it asserts on the pw:api debug output that this PR reworks — it's never flaked before.

Details

Caused by this PR

  • [installation tests] › playwright-packages-install-behavior.spec.ts:99 › @playwright/test should work — fails identically on ubuntu-latest, windows-latest, and macos-latest. The test runs npx playwright test failing.spec.js with DEBUG=pw:api and asserts expect(result4).toContain('expect.toHaveText started'). The fixture's failing assertion is expect(page.locator('span')).toHaveText(...), so the pw:api line is driven entirely by how the matcher's apiName is derived. This PR is exactly that change — it stops the test runner from plumbing apiName into the call and instead infers it from the stack, falling back to the explicit title for internal methods like locator._expect. Consistent failure on every OS with no browser involved, plus a 0-failure history across 2929 runs in the test-results DB, points at the apiName derivation rather than infra. Worth checking that the inferred name for expect matchers still comes out as expect.toHaveText.

Pre-existing flake / infra

  • [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate and [firefox] › mcp/cli-mouse.spec.ts:38 › mousewheel — both pre-existing flakes on SHAs unrelated to this PR (annotate flips verdict across ~23 PRs; mousewheel fails on push runs predating this branch). Nothing this PR touches reaches the MCP screencast or mouse-wheel paths.
  • The 5 tests listed as flaky (browsercontext-page-event, video, chromium, page-emulate-media, page-goto) all passed on retry within this run and are unrelated to matcher-title plumbing.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [chrome] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation @mcp-windows-latest-chrome

8100 passed, 1311 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

4 flaky ⚠️ [chromium-library] › library/chromium/chromium.spec.ts:179 › serviceWorker(), and fromServiceWorker() work `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@realtime-time-library-chromium-linux`
⚠️ [firefox-page] › page/page-event-request.spec.ts:181 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-ubuntu-22.04-node20`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:827 › should update state on subsequent run `@windows-latest-node22`

51121 passed, 1208 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a look at the CI failures here.

🔴 One failure looks caused by this PR

playwright-packages-install-behavior.spec.ts:99 › @playwright/test should work fails on all three OSes and asserts on the pw:api output that this PR reworks. It has never failed before. The Firefox/Chrome MCP failures are pre-existing flakes.

Details

Across the three CI reports on this PR, one failure is caused by the change and the rest are known flakes.

Caused by this PR

  • [installation tests] › playwright-packages-install-behavior.spec.ts:99 › @playwright/test should work — fails identically on ubuntu-latest, windows-latest and macos-latest. It runs npx playwright test failing.spec.js with DEBUG=pw:api and asserts expect(result4).toContain('expect.toHaveText started') (line 117). This PR changes exactly how that pw:api name is produced: channelOwner.ts now falls back to the explicit title for internal methods (apiName.includes('._')), and the matchers pass this.title into locator._expect instead of the runner instrumentation plumbing apiName = expect.toHaveText. If the fallback title isn't the short expect.toHaveText form, that log line no longer matches. The test-results DB backs this up: 1 failure in 992 runs, and that single failure is this PR's SHA 3f1f652 — no prior history on any other SHA. Worth confirming the inferred name for expect matchers still renders as expect.toHaveText.

Pre-existing flake / infra

  • [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate — pre-existing flake: failed 45 of 3461 runs (~1.3%) across 45 SHAs and 30 PRs unrelated to this one.
  • [chrome] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation — same story: 10 failures over 3452 runs across 7 PRs.
  • [firefox] › mcp/cli-mouse.spec.ts:38 › mousewheel — rarer, but it fails where this PR can't be responsible: two push runs on unrelated SHAs predating this branch — beaf223 (2026-07-29) and 9a19b13 (2026-08-04) — plus this run.

None of the MCP screencast, annotation or mouse-wheel paths are touched by the matcher-title change, and the 5 tests marked flaky in "tests 1" all passed on retry within the run.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants