Skip to content

fix: improve agent assignment failure issue reports #41392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 1 commit into from
Jun 25, 2026
Merged

fix: improve agent assignment failure issue reports #41392

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions actions/setup/js/handle_agent_failure.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ function buildFailureMatchCategories(options) {
* @param {boolean} options.hasDailyAICExceeded
* @param {boolean} options.aiCreditsRateLimitError
* @param {boolean} options.maxAICreditsExceeded
* @param {boolean} options.hasAssignmentErrors
* @returns {string}
*/
function buildFailureIssueTitle(options) {
Expand All @@ -260,6 +261,7 @@ function buildFailureIssueTitle(options) {
if (options.hasMissingSafeOutputs) return `[aw] ${workflowName} produced no safe outputs`;
if (options.hasMissingTool) return `[aw] ${workflowName} is missing required tool`;
if (options.hasMissingData) return `[aw] ${workflowName} is missing required data`;
if (options.hasAssignmentErrors) return `[aw] ${workflowName} failed to assign agent`;

@github-actions github-actions Bot Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] hasAssignmentErrors is placed last in buildFailureIssueTitle — just before the generic fallback — even though buildFailureMatchCategories treats it as a high-priority category (second, after isTimedOut).

In practice this is safe because hasMissingSafeOutputs is only set when agentConclusion === "success", and assignment errors drive agentConclusion to "failure". But the ordering relies on a non-obvious invariant worth documenting inline.

💡 Suggested comment 
// hasAssignmentErrors is checked last among failure-conclusion paths:
// all checks above it (hasMissingSafeOutputs, etc.) only fire on agentConclusion === "success",
// so there is no priority conflict in the common case.
if (options.hasAssignmentErrors) return `[aw] ${workflowName} failed to assign agent`;

This makes the ordering explicit and guards against future flags that might share the "failure" conclusion slipping in ahead of this check.

@github-actions github-actions Bot Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasAssignmentErrors is the lowest-priority title condition, but the second-highest in buildFailureMatchCategories — assignment error titles will be silently suppressed when isTimedOut also fires.

💡 Impact and suggested fix

In buildFailureMatchCategories (line ~197), hasAssignmentErrors is checked immediately after isTimedOut. In buildFailureIssueTitle it is last — below isTimedOut, hasMissingSafeOutputs, hasMissingTool, and hasMissingData.

Concrete scenario: the job times out while GitHub is retrying agent assignment (bad token, rate limit, service blip). Both isTimedOut = true and hasAssignmentErrors = true. The filed issue title becomes:

[aw] <workflow> timed out

...instead of:

[aw] <workflow> failed to assign agent

The user debugs phantom timeouts instead of checking their GH_AW_AGENT_TOKEN — exactly the misdirection this PR was written to prevent. The failure body still contains the assignment error context, but a dismissive "timed out" title means many users won't read that far.

Suggested fix — move the check to before isTimedOut, since assignment failure is a root cause, not a symptom:

function buildFailureIssueTitle(options) {
  const { workflowName } = options;
  if (options.hasAssignmentErrors) return `[aw] ${workflowName} failed to assign agent`; // check root causes first
  if (options.hasDailyAICExceeded) return `[aw] ${workflowName} exceeded daily AI credits budget`;
  // ... rest unchanged
}

Also add a test case that sets both hasAssignmentErrors: true and isTimedOut: true, asserting the title is "failed to assign agent", so the priority intent is locked in by CI.

return `[aw] ${workflowName} failed`;
}

Expand Down Expand Up @@ -2058,9 +2060,9 @@ function buildAssignmentErrorsContext(assignmentErrors) {
}

context += "\nTo resolve this, verify the agent token and Copilot access configuration:\n";
context += "- Configure a valid `GH_AW_AGENT_TOKEN` with `issues: write` and `pull-requests: write` plus active Copilot entitlement\n";
context += "- If your org supports it, add `permissions: { copilot-requests: write }` to use org inference without a personal token\n";
context += "- Docs: https://github.github.com/gh-aw/reference/engines/#github-copilot-default\n\n";
context += "- Configure a valid `GH_AW_AGENT_TOKEN` as a fine-grained PAT with **Agent tasks: read and write** permission (GitHub App installation tokens are not supported)\n";
context += "- Ensure Copilot coding agent is enabled for this repository and a Copilot Business or Enterprise subscription is active\n";
context += "- Docs: https://github.github.com/gh-aw/reference/copilot-cloud-agent/#authentication\n\n";

return context;
}
Expand Down Expand Up @@ -2990,6 +2992,7 @@ async function main() {
hasDailyAICExceeded,
aiCreditsRateLimitError,
maxAICreditsExceeded,
hasAssignmentErrors,
});
const failureCategories = buildFailureMatchCategories({
agentConclusion,
Expand Down
7 changes: 5 additions & 2 deletions actions/setup/js/handle_agent_failure.test.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ describe("handle_agent_failure", () => {
hasDailyAICExceeded: false,
aiCreditsRateLimitError: false,
maxAICreditsExceeded: false,
hasAssignmentErrors: false,
};

const cases = [
Expand All @@ -103,6 +104,7 @@ describe("handle_agent_failure", () => {
{ flag: "hasMissingSafeOutputs", expected: "[aw] Test Workflow produced no safe outputs" },
{ flag: "hasMissingTool", expected: "[aw] Test Workflow is missing required tool" },
{ flag: "hasMissingData", expected: "[aw] Test Workflow is missing required data" },
{ flag: "hasAssignmentErrors", expected: "[aw] Test Workflow failed to assign agent" },
];

it.each(cases)("returns expected title for isolated $flag", ({ flag, expected }) => {
Expand Down Expand Up @@ -1339,8 +1341,9 @@ describe("handle_agent_failure", () => {
expect(result).toContain("Issue #42 (agent: copilot): Bad credentials");
expect(result).toContain("PR #7 (agent: copilot): copilot coding agent is not available for this repository");
expect(result).toContain("GH_AW_AGENT_TOKEN");
expect(result).toContain("copilot-requests: write");
expect(result).toContain("https://github.github.com/gh-aw/reference/engines/#github-copilot-default");
expect(result).toContain("Agent tasks: read and write");
expect(result).not.toContain("copilot-requests: write");

@github-actions github-actions Bot Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Only copilot-requests: write is asserted absent; the two other removed strings aren't covered by a negative assertion.

The implementation also deleted issues: write and pull-requests: write from the guidance, but no not.toContain assertion guards those. A future regression that re-introduces either string would slip past the test suite.

💡 Suggested additions 
expect(result).not.toContain('copilot-requests: write');
expect(result).not.toContain('issues: write');
expect(result).not.toContain('pull-requests: write');

All three strings were removed from the implementation; all three deserve a guard.

expect(result).toContain("https://github.github.com/gh-aw/reference/copilot-cloud-agent/#authentication");
});
});

Expand Down
6 changes: 3 additions & 3 deletions docs/src/content/docs/reference/copilot-cloud-agent.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,10 @@ Both safe outputs require a fine-grained PAT. The default `GITHUB_TOKEN` lacks t

The required token type and permissions depend on whether you own the repository or an organization owns it.

1. **Create the PAT** with **Repository permissions**: Actions, Contents, Issues, Pull requests (all Write).
1. **Create the PAT** with **Repository permissions**: Actions, Contents, Agent tasks (read and write).

- [User-owned repositories](https://github.com/settings/personal-access-tokens/new?name=GH_AW_AGENT_TOKEN&description=GitHub+Agentic+Workflows+-+Agent+assignment&actions=write&contents=write&issues=write&pull_requests=write): Resource owner = your user account; Repository access = "Public repositories" or specific repos
- [Organization-owned repositories](https://github.com/settings/personal-access-tokens/new?name=GH_AW_AGENT_TOKEN&description=GitHub+Agentic+Workflows+-+Agent+assignment&actions=write&contents=write&issues=write&pull_requests=write): Resource owner = the organization; Repository access = specific repositories that will use the workflow
- [User-owned repositories](https://github.com/settings/personal-access-tokens/new?name=GH_AW_AGENT_TOKEN&description=GitHub+Agentic+Workflows+-+Agent+assignment&actions=write&contents=write&agent_tasks=write): Resource owner = your user account; Repository access = "Public repositories" or specific repos
- [Organization-owned repositories](https://github.com/settings/personal-access-tokens/new?name=GH_AW_AGENT_TOKEN&description=GitHub+Agentic+Workflows+-+Agent+assignment&actions=write&contents=write&agent_tasks=write): Resource owner = the organization; Repository access = specific repositories that will use the workflow

2. Add to repository secrets:

Expand Down
Loading