Skip to content

ci(deploy): fail-closed serves-this-commit assert in verify-deploy.sh - #53

Merged
vreshch merged 1 commit into
masterfrom
fix/deploy-commit-assert
Aug 12, 2026
Merged

ci(deploy): fail-closed serves-this-commit assert in verify-deploy.sh#53
vreshch merged 1 commit into
masterfrom
fix/deploy-commit-assert

Conversation

@vreshch

@vreshch vreshch commented Aug 12, 2026

Copy link
Copy Markdown
Member

Part of the estate-wide deploy-verification standardization.

Gap

The only post-deploy check was curl https://${SITE_FQDN}/ | grep -q "Agentage Design System" - fail-open. Compose uses order: start-first + failure_action: rollback, so on a failed rollout Swarm keeps the OLD container serving; the grep still matches and the deploy goes green.

Changes

  • scripts/verify-deploy.sh (new, same shape as auth/dashboard): reads SITE_FQDN + COMMIT_SHA from env, asserts /health reports this commit (30 x 10s, prefix match, ::error:: + exit 1 on timeout), then keeps the render grep as a secondary criterion. Commit-first on purpose.
  • deploy.yml: post-deploy step reduces to bash scripts/verify-deploy.sh with SITE_FQDN + COMMIT_SHA: ${{ github.sha }}.
  • deploy.yml smoke job: also asserts the built image's /health commit equals github.sha, so a build that fails to bake COMMIT_SHA fails before prod rather than as an unexplained post-deploy timeout.
  • CLAUDE.md: one line on where verification lives and why the grep alone was fail-open.

No /health work was needed - #51 already bakes COMMIT_SHA into health.json and serves it as an exact-match nginx location. Build-arg was already wired.

How verified

  • Built the image locally (--build-arg COMMIT_SHA=abc123...de), ran it, GET /health returned {"success":true,"data":{"status":"ok","service":"agentage-ds","commit":"abc123...de","buildTime":"..."}}; the grep -o extraction used by both the script and the smoke job returned that exact sha. Container healthcheck went healthy.
  • Ran scripts/verify-deploy.sh against live prod with master's real sha -> both criteria PASS, exit 0.
  • Fail-closed paths (retries shortened for the test): prod serving a different commit -> ::error:: + exit 1 while the render grep still passed (this is exactly the old fail-open case); unreachable host -> exit 1; missing COMMIT_SHA -> exit 1.
  • bash -n + shellcheck clean; deploy.yml parses as YAML; prettier --check scripts .github/workflows/deploy.yml clean (the dir walk CI runs skips .sh). No TS/JS touched.

The only post-deploy check was a content grep, which the OLD container answers just as happily on a failed rollout - deploy went green either way. The showcase already bakes COMMIT_SHA into /health (#51), so assert that instead, commit-first, then keep the render grep as a secondary criterion. Extracted to scripts/verify-deploy.sh matching auth/dashboard.
@vreshch
vreshch marked this pull request as ready for review August 12, 2026 23:45
@vreshch
vreshch merged commit 676986e into master Aug 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant