Skip to content

fix: create-release noop — semver step outputs unavailable during prompt interpolation - #222

Merged
pelikhan merged 1 commit into
mainfrom
copilot/investigate-safeoutputs-create-release
Jul 27, 2026
Merged

fix: create-release noop — semver step outputs unavailable during prompt interpolation#222
pelikhan merged 1 commit into
mainfrom
copilot/investigate-safeoutputs-create-release

Conversation

Copilot AI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Pre-agent steps: are compiled into the agent job, but prompt interpolation happens in the activation job (which runs first). Any ${{ steps.<id>.outputs.* }} reference in the prompt body is therefore always empty — the step hasn't run yet in that job context.

create-release.md computed next_version / previous_tag in a semver pre-step and used both in the prompt. The agent received a prompt with literal empty strings and correctly called noop.

Changes

  • Remove semver pre-step — its outputs can never reach the prompt; the step was dead code from the prompt's perspective
  • Rewrite task body — agent now discovers the previous tag via git describe --tags --abbrev=0 and computes the bump itself, guided by ${{ github.event.inputs.bump }} (a workflow dispatch input, properly interpolated at activation time)
  • Recompile lock fileGH_AW_STEPS_SEMVER_OUTPUTS_* env vars removed from activation job steps

Before (prompt received by agent):

You are preparing release **v**
(a **patch** bump from ``).
…
Call `noop` if … or if a tag `v` already exists.

After: agent runs git describe --tags itself and computes the version before calling create_release.

…rom git

The `semver` step was in the `steps:` block (compiled into the agent job),
but its outputs were referenced in the prompt template which is built in the
activation job. Since `steps.semver.outputs.*` is always empty in the
activation job context, the agent received empty version strings and called
`noop` instead of `create_release`.

Fix: remove the semver pre-step and update the task to have the agent
compute the next version itself from `git describe --tags` plus the
`${{ github.event.inputs.bump }}` input (which IS properly interpolated
as an event input).

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title fix: create-release agent calls noop because semver step outputs are empty in prompt fix: create-release noop — semver step outputs unavailable during prompt interpolation Jul 27, 2026
Copilot AI requested a review from pelikhan July 27, 2026 07:51
@pelikhan
pelikhan marked this pull request as ready for review July 27, 2026 07:53
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@pelikhan
pelikhan merged commit b3cb206 into main Jul 27, 2026
4 checks passed

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs — the root cause is correctly identified and the fix is sound. Two suggestions to harden the agent's version computation.

📋 Key Themes & Highlights

Key Themes

  • Root cause correctly addressed ✅ — The pre-step outputs were genuinely unreachable at prompt-interpolation time (activation job vs. agent job). Removing them and moving the discovery into the agent prompt is the right fix.
  • Determinism risk — The new Step 1 asks the LLM to compute semver mentally from prose rules. The removed pre-step did this deterministically in shell; the same shell snippet should be provided as an executable block in the prompt so the agent runs it rather than reasons about it.
  • Ambiguous noop guard — "if the computed tag already exists" is vaguer than the old v${{ steps.semver.outputs.next_version }}. Worth tightening the wording to reference the variable name the agent is expected to carry forward from Step 1.

Positive Highlights

  • ✅ Clear PR description with a concrete before/after — easy to understand what was broken and why
  • ✅ Lock file regenerated consistently with the source .md changes
  • ${{ github.event.inputs.bump }} is correctly identified as safe to interpolate (workflow dispatch input, available at activation time)
  • ✅ Step numbering updated throughout — no orphaned references

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 33.8 AIC · ⌖ 6.06 AIC · ⊞ 6.3K
Comment /matt to run again

Comments that could not be inline-anchored

.github/workflows/create-release.md:92

[/diagnosing-bugs] Relying on the LLM to perform semver arithmetic is a latent correctness risk — e.g., the agent could misparse v0.10.0 as v0.1.0 or mishandle pre-release suffixes.

<details>
<summary>💡 Suggestion: give the agent a shell snippet to run instead of computing mentally</summary>

The removed pre-step's logic was correct; the only problem was where it ran. Provide a copy of that shell logic as an executable block in Step 1 so the agent runs it and reads the output rather t…

.github/workflows/create-release.md:151

[/diagnosing-bugs] The noop guard now says "if the computed tag already exists" — but if the agent makes a computation error in Step 1, it may check for the wrong tag and skip a duplicate-release guard. Tying the guard to a shell-computed value (see Step 1 suggestion) makes it unambiguous.

<details>
<summary>💡 Suggestion</summary>

Be explicit: store the computed version in a variable at Step 1 and reference it by name in the guard:

Call `noop` if the commit list is empty or if tag…

</details>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants