Add pre-deploy test gate and fix Pages two-publisher race - #887
Merged
Conversation
Mirrors the pattern used by Azure/ai-app-templates: - New reusable workflow .github/workflows/website-test.yml runs unit tests (jest), builds the Docusaurus site, and runs Playwright e2e tests against the built artifact. - release.yml gains a test-website job that runs before sync, so a failing test blocks the deploy. - Removes .github/workflows/test-deploy.yml (now redundant: website-test.yml triggers on pull_request/push to main with the same paths filter and runs a superset of its checks). - Drops the now-duplicate `npm test` step from the deploy job (already covered by the gate). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a stronger pre-release validation gate for the Docusaurus website by adding a reusable website test workflow (unit + build + Playwright e2e) and wiring it into the manual release pipeline, replacing the previous PR-only build/test workflow.
Changes:
- Added a new reusable workflow (
website-test.yml) that runsnpm ci,npm test,npm run build, and Playwright e2e tests, uploading the Playwright report on failure. - Updated the release workflow to run the website test workflow as a prerequisite before proceeding with sync/deploy, and removed the redundant
npm testfrom deploy. - Removed the old
test-deploy.ymlworkflow.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.github/workflows/website-test.yml |
New reusable CI workflow that runs unit tests, build, and Playwright e2e tests for the website. |
.github/workflows/release.yml |
Adds a website test gate job and makes the sync job depend on it; removes redundant deploy-time unit test step. |
.github/workflows/test-deploy.yml |
Deletes the prior PR-only workflow in favor of the new consolidated website test workflow. |
Victor Vazquez (vhvb1989)
approved these changes
May 20, 2026
Victor Vazquez (vhvb1989)
approved these changes
May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix #884
Summary
Hardens the release pipeline by:
actions/deploy-pagesthat produced the recent outage where the live site rendered the rawREADME.mdinstead of the gallery.Why
Releases were dispatched manually with no guarantee the site built, passed tests, or rendered correctly. The recent outage exposed a second problem: Pages was configured with
build_type: legacy(source =gh-pagesbranch), so every push to that branch auto-triggered GitHub's built-inpages-build-deployment(Jekyll) in parallel withactions/deploy-pages. Jekyll rendered the rootREADME.md(no_config.yml/index.html), finished ~2 minutes afterdeploy-pages, and clobbered the gallery output. The workflow happily reported success either way.This PR closes both gaps in one shot: a test gate catches broken builds before they reach production, and switching Pages to "GitHub Actions" as the source removes the auto-trigger entirely so
actions/deploy-pagesbecomes the only publisher.What changes
New reusable workflow
.github/workflows/website-test.ymlnpm ci+npm test+npm run build(the lightweight gate for PRs and pushes).npx playwright install --with-deps chromium+npx playwright test) are gated byif: github.event_name == 'workflow_call' || github.event_name == 'workflow_dispatch', so the heavy browser suite only runs as part of a release (called fromrelease.yml) or when explicitly dispatched — not on every PR.pull_request/pushonmain(paths-filtered towebsite/**and the workflow itself), plusworkflow_call/workflow_dispatch.ref: ${{ github.sha }}for defense-in-depth (immutable input to the test job)..github/workflows/release.ymlguardjob withpermissions: {}that fails fast ifgithub.ref != refs/heads/main, preventing accidental release runs from feature branches.test-websitejob (callswebsite-test.yml) gated byguard.deploynowneeds: [guard, test-website], so the deploy chain only proceeds after both pass.npm teststep fromdeploy(covered by the test gate).syncjob (see below).ref: ${{ github.sha }}..github/workflows/sync-gh-pages.yml(removed)Previously force-pushed the website build to the
gh-pagesbranch. That push is what triggered Jekyll. With Pages source switched to "GitHub Actions", there is nogh-pagesbranch to keep in sync —actions/deploy-pagespublishes the artifact directly. Deleting the workflow removes the auto-trigger..github/workflows/test-deploy.yml(removed)Superseded by
website-test.yml, which fires on the same triggers and runs a superset of its checks.Action SHA refresh
actions/upload-pages-artifactv4 → v5 (fc324d3547104276b827a68afc52ff2a11cc49c9)actions/deploy-pagesv4 → v5 (cd2ce8fcbc39b97be8ca5fce6e763baed58fa128)actions/upload-artifactv7.0.0 → v7.0.1 (043fb46d1a93c77aae656e7c1c64a875d1fc6a0a)actions/github-scriptv9.0.0 SHA refresh (d746ffe35508b1917358783b479e04febd2b8f71) indiscover-templates-extensions.ymlandextension-submission.ymlThe v4→v5 upgrade was blocked by an artifact format mismatch (upload v5 uses immutable artifacts under
upload-artifact@v7, whichdeploy-pages@v4couldn't consume). Moving both to v5 in lockstep resolves it and unblocks the revert from #883.Playwright spec changes
website/e2e/gallery-functionality.spec.ts— 7 deploy-gate tests covering: homepage hero, a healthy template count (≥50), card rendering, narrowing search ("azure"), empty-state messaging, Language-section tag filter (Python), and clear-search restoration. Reads the total from the[role="status"]"Viewing N templates" live region rather than counting.fui-Card(which is page-bound to 20 by pagination).SearchBoxonly updates the URL ononSearch(Enter) — added.press("Enter")after.fill(...)ingallery-filters.spec.tsandhomepage.spec.ts..clear()doesn't fire Fluent UI's clear-X handler — replaced withfill("") + press("Enter")ingallery-filters.spec.ts.getting-started.spec.ts: corrected hero text ("Ship in minutes" — was "Ship to Azure"); addedexact: trueon the "Pick a template" / "Deploy to Azure" headings to avoid strict-mode collision with the hero h1.website/.gitignore/playwright-report,/test-results,/playwright/.cacheto keep local Playwright artifacts out of the repo.!*.pngto override the root-level*.pngrule, sowebsite/static/**/*.pngand any future visual-regression baselines are tracked.Removed an accidentally-committed file
website/playwright-report/index.html(526KB report bundle) was committed in an earlier revision of this branch. Deleted.Operational steps still required (cannot be done from PR)
github-pages→ update Deployment branches fromgh-pagestomain.Without these two clicks, the workflow change alone won't take effect.
Validation
npm run serve(Docusaurus production build).pull_request) runs only the lightweight gate (npm ci+npm test+npm run build).workflow_callfromrelease.yml) runs the full Playwright suite as the deploy gate.Notes for reviewers
Test deploymentwill need to be renamed to whatever job namewebsite-test.ymlemits once this lands — call out in a follow-up.