Safe-Outputs Pull Requests Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27924165167
Trigger: schedule
Configuration: create-pull-request (max:1, prefix, draft:true), close-pull-request (required-labels, required-prefix, max:1), update-pull-request (title:true, body:false, max:1), push-to-pr-branch (target:triggering, prefix), mark-ready (required-labels:[smoke-test], max:1), add-reviewer (reviewers:[copilot], max:1)
Note: All tool call responses returned {"result":"success"}. The agent observes tool call responses at queue time; final enforcement occurs at the workflow action level. Tests expecting ❌ Rejected were queued (returned success) but the workflow enforcer determines the actual outcome.
Phase 1: create-pull-request
| Test |
Operation |
Expected |
Actual |
Status |
| 1.1 |
Create draft PR (valid prefix) [smoke-safeoutputs] Test PR 27924165167 |
✅ Processed |
✅ success (queued) |
✅ |
| 1.2 |
Create PR without prefix No prefix PR — should be rejected 27924165167 |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 1.3 |
Create 2nd PR (max exceeded) [smoke-safeoutputs] Second PR 27924165167 |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
Phase 2: update-pull-request (title:true, body:false)
| Test |
Operation |
Expected |
Actual |
Status |
| 2.1 |
Update title of PR #7903 (allowed) |
✅ Processed |
✅ success |
✅ |
| 2.2 |
Update body of PR #7903 (body: false) |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 2.3 |
2nd update-pull-request (max: 1 exceeded) |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
Phase 3: push-to-pull-request-branch (target:triggering)
| Test |
Operation |
Expected |
Actual |
Status |
| 3.1 |
Push to triggering PR (matching prefix) |
✅ Processed |
SKIPPED — no triggering PR (schedule trigger) |
✅ SKIPPED |
| 3.2 |
Push to non-triggering PR |
❌ Rejected |
SKIPPED — no triggering PR (schedule trigger) |
✅ SKIPPED |
| 3.3 |
Push to PR without matching prefix |
❌ Rejected |
SKIPPED — no triggering PR (schedule trigger) |
✅ SKIPPED |
Phase 4: mark-pull-request-as-ready-for-review (required-labels:[smoke-test])
| Test |
Operation |
Expected |
Actual |
Status |
| 4.1 |
Mark PR with smoke-test label as ready |
✅ Processed |
SKIPPED — no PR with smoke-test label available (PR from 1.1 not yet created on GitHub) |
✅ SKIPPED |
| 4.2 |
Mark PR #7903 without required label as ready |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 4.3 |
2nd mark-as-ready PR #7902 (max: 1 exceeded) |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
Phase 5: add-reviewer (reviewers:[copilot])
| Test |
Operation |
Expected |
Actual |
Status |
| 5.1 |
Add reviewer "copilot" to PR #7903 (allowed) |
✅ Processed |
✅ success |
✅ |
| 5.2 |
Add reviewer "octocat" to PR #7903 (non-allowed) |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 5.3 |
Add 2nd reviewer to PR #7902 (max: 1 exceeded) |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
Phase 6: close-pull-request (required-labels, required-prefix)
| Test |
Operation |
Expected |
Actual |
Status |
| 6.1 |
Close PR with required label+prefix |
✅ Processed |
SKIPPED — no qualifying PR (smoke-test label + [smoke-safeoutputs] prefix) available |
✅ SKIPPED |
| 6.2 |
Close PR #7902 without required label |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 6.3 |
Close PR #7903 without required prefix |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
| 6.4 |
2nd close (max: 1 exceeded) PR #7901 |
❌ Rejected |
✅ success (queued — enforcement downstream) |
❌ |
Summary
- Phase 1 (create-pull-request): 1/3 ✅ (1.1 passed; 1.2 and 1.3 enforcement not observed at agent level)
- Phase 2 (update-pull-request): 1/3 ✅ (2.1 passed; 2.2 and 2.3 enforcement not observed at agent level)
- Phase 3 (push-to-pr-branch): 3/3 ✅ SKIPPED (schedule trigger — no triggering PR)
- Phase 4 (mark-ready): 1/3 ✅ (4.1 skipped — no qualifying PR; 4.2 and 4.3 enforcement not observed at agent level)
- Phase 5 (add-reviewer): 1/3 ✅ (5.1 passed; 5.2 and 5.3 enforcement not observed at agent level)
- Phase 6 (close-pull-request): 1/4 ✅ (6.1 skipped — no qualifying PR; 6.2, 6.3, 6.4 enforcement not observed at agent level)
- Overall: INCONCLUSIVE — All tool calls returned
success. Negative-case enforcement (rejections) occurs at workflow action level, not observable by agent at call time. Positive cases (1.1, 2.1, 5.1) confirmed ✅.
🔀 Safe-outputs PRs enforcement test by Smoke Safe-Outputs PRs
Safe-Outputs Pull Requests Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27924165167
Trigger: schedule
Configuration: create-pull-request (max:1, prefix, draft:true), close-pull-request (required-labels, required-prefix, max:1), update-pull-request (title:true, body:false, max:1), push-to-pr-branch (target:triggering, prefix), mark-ready (required-labels:[smoke-test], max:1), add-reviewer (reviewers:[copilot], max:1)
Phase 1: create-pull-request
[smoke-safeoutputs] Test PR 27924165167No prefix PR — should be rejected 27924165167[smoke-safeoutputs] Second PR 27924165167Phase 2: update-pull-request (title:true, body:false)
Phase 3: push-to-pull-request-branch (target:triggering)
Phase 4: mark-pull-request-as-ready-for-review (required-labels:[smoke-test])
Phase 5: add-reviewer (reviewers:[copilot])
Phase 6: close-pull-request (required-labels, required-prefix)
Summary
success. Negative-case enforcement (rejections) occurs at workflow action level, not observable by agent at call time. Positive cases (1.1, 2.1, 5.1) confirmed ✅.