test: nightly Shopify live contract suite against the dev store#967
Merged
Conversation
…-only by construction) Layer 1 of the Shopify test plan: a separate jest project (checkin-app/shopify-live/, npm run test:shopify-live) that drives the REAL lib/shopify.ts functions — createShopifySingleVariantProgram, adjustProgramInventory, mintMemberDiscountCode, getAccessToken — against the REAL dev store, pinning the Admin-API contracts the capacity model depends on and that mocks structurally cannot verify: - product-inventory: active product, managed deny-policy variant, initial inventory = maxParticipants, RELATIVE ±1 adjusts (the #930 hold ops) - discount: minted rule is fixed_amount + 'each' (the multi-child overcharge class), variant-scoped, single-use, ~48h window - api-version-canary: pinned SHOPIFY_API_VERSION echoed back with no deprecation header — version sunsets become a red nightly Dev-store-only by a triple guard (shopify-live/guard.ts, pure and unit-tested in the NORMAL CI suite): production domain denylisted unconditionally; SHOPIFY_STORE_DOMAIN must equal the human-provisioned SHOPIFY_LIVE_ALLOWED_DOMAIN repo variable (two-edit retarget); *.myshopify.com only. prisma/email are the only mocks (failure reports must not write a DB or email the board from a test run). Cleanup: citest-tagged titles + reserved PRG9999999xx- discount range; track-on-create + afterAll deletes; scripts/shopify-live-janitor.ts sweeps before (24h cutoff) and after (0h, always()) each workflow run. Workflow .github/workflows/shopify-live.yml: nightly + dispatch, serialized, DORMANT until repo vars/secrets are provisioned (SHOPIFY_LIVE_ENABLED, SHOPIFY_LIVE_ALLOWED_DOMAIN, SHOPIFY_LIVE_CLIENT_ID/SECRET — see the design doc §5). Verified without real creds: guard unit tests (7) green in the normal suite; the live suite fails fast with the provisioning message when creds are absent, and with fake creds it passes the guard and reaches the real dev store's token endpoint (Shopify 400) — the only unexercised link is real credentials, which is the CI provisioning step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…them up The 'Run Tests' job runs test:coverage, whose CLI --testPathIgnorePatterns REPLACES the config-level ignore list, so the jest.config.js exclusion never applied there and the live suites ran (and correctly fail-fast guarded) in CI. Ignore-lists are fragile against any CLI override; the root fix is structural: *.shopify-live.ts carries no .test suffix, so no default testMatch or overridden ignore list can ever match it — only jest.shopify-live.config.js's explicit testMatch finds the suite. Drops the now-unneeded jest.config.js ignore entry; AGENTS.md notes the naming rationale. Verified: --listTests shows 0 live files for both the default run and the exact test:coverage CLI invocation, 3 for the live config; creds-absent fail-fast unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
thpr
approved these changes
Jul 8, 2026
✅ Deployed to devCommit
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Layer 1 of the Shopify test plan: a live contract suite that runs the app's real
lib/shopify.tsfunctions against the real dev store nightly, pinning the Admin-API semantics our money paths depend on — the things mocked tests structurally cannot verify.product-inventorydeny-policy variant at the right price; initial inventory =maxParticipants; relative ±1 adjusts (the #930 scholarship hold/release ops)discountfixed_amount+eachallocation (the multi-child overcharge class from the #930 review), entitled-variant-scoped, single-use, ~48h windowapi-version-canarySHOPIFY_API_VERSIONis echoed back (x-shopify-api-version) with no deprecation header — version sunsets become a red nightly instead of silent behavior driftDev store only, by construction
shopify-live/guard.ts— pure, and unit-tested in the normal CI suite (so the guard is verified even though the live suite never runs in CI):SHOPIFY_STORE_DOMAIN(pinned in the workflow) must equal the human-provisionedSHOPIFY_LIVE_ALLOWED_DOMAINrepo variable — retargeting takes two deliberate edits in two places;*.myshopify.comonly.Hygiene
citest-titles + a reservedPRG9999999xx-discount range; track-on-create +afterAlldeletes;scripts/shopify-live-janitor.tssweeps before each run (24h cutoff) and after every run — pass or fail — with a 0h cutoff. Suite is serialized (--runInBand, ~25 Admin calls/run against the 2 req/s budget) and excluded from unit/CI/pre-commit runs via the flow-tests convention.Provisioning (one-time, then the workflow wakes up)
Repo variables
SHOPIFY_LIVE_ENABLED=true+SHOPIFY_LIVE_ALLOWED_DOMAIN=treehouse-dev-4folhtgx.myshopify.com; secretsSHOPIFY_LIVE_CLIENT_ID/SHOPIFY_LIVE_CLIENT_SECRET(dev app; scopeswrite_products,write_inventory,write_price_rules,read_locations). Firstworkflow_dispatchrun is the acceptance test. Nightly red is notification-only until it earns a week of green.Verified without real creds
Guard unit tests (7) green in the normal suite; creds-absent → the live suite fails fast with the provisioning message; fake creds → passes the guard, reaches the real dev store's token endpoint, Shopify 400. The only unexercised link is real credentials — exactly what provisioning supplies.
Design doc:
checkin-app/docs/designs/SHOPIFY_LIVE_TESTS.md(includes the deferred layers: ops-dev webhook E2E smoke, config-drift reconciliation, and the #955 archive-status contract once it merges).🤖 Generated with Claude Code