Skip to content

fix(tests): prepare AgentPlayground CI cutover#1560

Open
jamesadevine wants to merge 1 commit into
mainfrom
jamesadevine/fix-ci-misses
Open

fix(tests): prepare AgentPlayground CI cutover#1560
jamesadevine wants to merge 1 commit into
mainfrom
jamesadevine/fix-ci-misses

Conversation

@jamesadevine

@jamesadevine jamesadevine commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Prepares the repository side of the AgentPlayground CI migration from #1498
and #1504.

  • Adds opt-in, fast-forward-only trigger E2E synchronization from the
    orchestrator's checked-out main commit to ado-aw-mirror.
  • Fails before victim scenarios on mirror divergence, verifies the remote SHA,
    and keeps the ADO bearer out of argv and git config.
  • Corrects the smoke failure reporter to query ADO-safe definition names and
    use jamesadevine/ado-aw-issues while canonical issue credentials are
    unavailable.
  • Updates the smoke, executor, and trigger E2E deployment handoffs for the
    official GitHub connection, replacement definitions, permissions, secrets,
    and mirror ownership.
  • Removes stale .gitattributes entries for the 23 deleted per-tool smoke
    fixtures.
  • Regenerates the reporter lock with ado-aw v0.45.1. Its AWF change to
    0.27.9 is intentional and follows fix(compile): downgrade AWF for host access #1526, which restored host-access
    compatibility.

Validation

  • cargo test --bin ado-aw enable
  • cargo test --test compiler_tests test_smoke_failure_reporter_uses_registered_ado_names_and_staging_repo
  • cargo run -- check tests\safe-outputs\smoke-failure-reporter.lock.yml
  • npm run typecheck
  • Trigger mirror + runner tests: 17 passed
  • npm run build:trigger-e2e
  • Full ado-script suite: 538 passed; the existing Windows-only
    executor-e2e/__tests__/runner.test.ts node-shebang spawn case remains the
    sole failure, matching the pre-existing baseline documented in test(trigger-e2e): add deterministic gate/synth-PR trigger-condition E2E suite #1504.

Operational handoff after merge

No Azure DevOps definitions or permissions are changed by this PR. After
merge, the replacement-first cutover will create ado-aw-mirror, five smoke
definitions, one executor definition, and the trigger victim/orchestrator.
The staging repository still needs ado-aw-smoke and
trigger-e2e-failure labels before failure filing is enabled.

Add fail-closed trigger mirror synchronization, align smoke failure reporting with deployed ADO names and the staging issue repository, and update the operational handoff.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 0fe13dc9-ae18-4e54-b587-a53f5afb28f2
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Looks good — well-structured, security-conscious, and thoroughly tested. One version change worth confirming.

Findings

⚠️ Suggestions

  • smoke-failure-reporter.lock.yml line 236 — AWF version downgrade: AWF_VERSION goes from 0.27.280.27.9 (a downgrade by patch number). The PR description mentions "replacement definitions" and a migration context, so this may be intentional. Just worth a double-check that 0.27.9 is the correct target for the new AgentPlayground service connection setup and not an accidental revert.

  • mirror.ts:162diagnostic() redacts token but not URL form: The bearer token is redacted from diagnostic output via replaceAll(secret, "***") — good. However, if the ADO URL passed to push/ls-remote ever gets echoed in stderr (e.g., from a git error), the token won't be embedded there (it's in env, not argv) so this is fine as-is. Just noting the defense-in-depth is already correct.

✅ What Looks Good

  • Token out of argv: Bearer auth is passed via GIT_CONFIG_VALUE_0 env var, not as a URL credential or command argument. The test explicitly asserts push?.args.join(" ")).not.toContain("secret-token"). Correct pattern.
  • Fast-forward-only + post-push SHA verification: The push --porcelain without --force fails closed on non-FF, and the subsequent ls-remote verification confirms the remote HEAD matches — solid correctness guarantee.
  • Shallow-checkout guard: Refusing to sync a shallow checkout prevents silent partial-history mirroring.
  • cleanVar macro guard: Stripping unexpanded $(VAR) ADO macro tokens before using them is a good defensive practice in pipeline-executed scripts.
  • Test coverage: The 7 test cases in mirror.test.ts cover all the important failure modes (non-main branch, shallow clone, non-FF rejection, SHA mismatch, unset victim repo).
  • Rust test: test_smoke_failure_reporter_uses_registered_ado_names_and_staging_repo is a clean content-contract test that will catch accidental definition-name regressions.

Generated by Rust PR Reviewer for #1560 · 42.9 AIC · ⌖ 5.73 AIC · ⊞ 6.2K ·

@jamesadevine

Copy link
Copy Markdown
Collaborator Author

Confirmed the AWF change is intentional: the current compiler constant is 0.27.9 (src/compile/common.rs), set by #1526 to restore host-access compatibility. Regenerating the reporter lock from v0.44.1 to v0.45.1 picked up that canonical version; no follow-up code change is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant