feat(qa): full-surface coverage, contract-fed vision judge, nightly design loop#577
Merged
Conversation
…nightly loop
Roadmap items 1-3 from the agent-native QA plan:
1. Coverage — the contract gains replay-miss (unknown /r/{hash} share
links must degrade into the workspace with an honest banner) and
mobile-entry-redirect states, with new runner vocabulary (per-state
viewport, expectUrlPattern, expect/forbidTestId). Probing the live
bundle showed the standalone mobile shell no longer mounts since the
one-surface contraction, so live-smoke-mobile — red against current
production while living outside CI, the exact #567 failure mode — is
rewritten to assert the one-surface reality. 7/7 runner, 6/6 mobile
smoke against the served bundle.
2. Vision judge grounding — the Gemini QA judge prompt now receives
every contract clause as CONTRACT-VERIFIED INVARIANTS ground truth,
so paid vision passes stop re-litigating what CI already proves and
findings that contradict a verified clause classify as misreads.
3. Cadence — nightly-design-loop.yml builds main, executes every
surface contract (fails on drift), captures the 4-variant screenshot
set, and uploads both as reviewable artifacts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HomenShum
enabled auto-merge (squash)
July 17, 2026 21:16
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
✅ Dogfood Visual QA Gate: PASSED
ArtifactsDownload the Generated by Dogfood QA Gate |
|
Demo: walkthrough of the surfaces this PR changed is available as a workflow artifact ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The three remaining agent-native QA roadmap items, on top of #575/#576:
/r/{hash}share links (must degrade into the workspace with an honest banner) and the mobile entry redirect; runner vocabulary extended (per-stateviewport,expectUrlPattern,expect/forbidTestId). Live probing showed the standalone mobile shell no longer mounts since the one-surface contraction, solive-smoke-mobile— which was red against current production while living outside CI, the exact ScratchnodeEventsSurface.test.tsx is red on main and CI can't see it #567 failure mode — is rewritten to assert the one-surface reality.CONTRACT-VERIFIED INVARIANTSground truth; findings contradicting machine-proven clauses classify as misreads instead of bugs, cutting paid re-litigation and score variance.nightly-design-loop.ymlbuilds main nightly, executes every surface contract (fails on drift), captures the 4-variant screenshot set, and uploads both as reviewable artifacts.Verification
npx tsc --noEmit0 errors;node --checkclean on the QA script (caught and fixed a duplicate-import on first pass)🤖 Generated with Claude Code