Skip to content

Align eval-harness Python version with CI (3.14) - #498

Merged
joryirving merged 1 commit into
mainfrom
foreman/wl-misospace-pr-reviewer-action-496/issue-496
Aug 12, 2026
Merged

Align eval-harness Python version with CI (3.14)#498
joryirving merged 1 commit into
mainfrom
foreman/wl-misospace-pr-reviewer-action-496/issue-496

Conversation

@itsmiso-ai

Copy link
Copy Markdown
Contributor

What

Pinned Python version to 3.14 in eval-harness.yaml to match CI and resolve duplicate-key regression test failure.

Why

Version skew between local (3.9) and CI (3.x) caused the duplicate-key regression test to ship broken; pinning to 3.14 ensures consistent behavior.…

Fixes #496

Opened by foreman on review GO (workload wl-misospace-pr-reviewer-action-496).

Fixes #496

Change python-version in eval-harness.yaml from "3.12" to '3.14' to match
ci.yaml, and add a comment mirroring the version-skew rationale documented
in ci.yaml. The action pins were already SHA-pinned by Renovate PRs #491–493;
this closes the remaining interpreter gap.

Signed-off-by: Saffron <263493777+itsmiso-ai@users.noreply.github.com>
@itsmiso-ai
itsmiso-ai requested a review from joryirving as a code owner August 12, 2026 13:50

@its-saffron its-saffron Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Automated Review

Full PR review.

Analysis engine: dsv4f@https://litellm.jory.dev/v1 (openai) — primary route

Recommendation

Approve. This is a minimal, well-targeted fix that aligns the eval-harness workflow's Python interpreter with CI (3.14), directly satisfying the linked issue's primary acceptance criterion ("eval-harness.yaml pins the same Python version as ci.yaml ... matching literal"). The validate CI check passes on this commit, and repo history/grep confirm ci.yaml pins python-version: '3.14' as the reference. No blockers or material issues found.

Change-by-change findings

.github/workflows/eval-harness.yaml

  • python-version changed from "3.12" to '3.14', byte-for-byte the same literal as ci.yaml:24 (python-version: '3.14'), so the weekly scheduled eval run and CI now exercise the identical interpreter on the same pr_reviewer code. This was the one gap Renovate couldn't close (the Renovate PRs PR 491–493 already converted the workflow's actions to SHA-pinned v7 forms, which I confirmed are present in the current file at checkout 3d3c42e5..., 5fda3b95..., and 043fb46d...).
  • The new explanatory comment matches the spirit and content of the ci.yaml comment about the duplicate-key regression and the local 3.9 vs. CI 3.x skew; it's accurate and mirrors the accepted rationale from commit 112cd5a.
  • The quoted scalar ('3.14') matches the exact style used in ci.yaml, which aids the stated goal of keeping the two workflows visually consistent.
  • No security, secret-handling, or path concerns: this file only touches workflow interpreter pinning, uses the existing secrets.* references unchanged, and carries no risk flags.

Sources

  • Full contents of .github/workflows/eval-harness.yaml (already SHA-pinned actions v7 + new 3.14 pin)
  • git grep for 3.14/python-version: matches in both .github/workflows/ci.yaml:24 and .github/workflows/eval-harness.yaml:62, with identical co-located comments
  • Repository history: a5f5814 (this PR's head), 84f0e6c (eval harness CI workflow PR 488), 112cd5a (ci: catch duplicate YAML keys + pin CI Python to 3.14 PR 256), Renovate PRs PR 491–493 (action SHAs)
  • Issue PR 496 body (acceptance criteria and evidence)
  • CI check result for a5f5814: validate success

Standards Compliance

AGENTS.md contains no workflow-interpreter or action-pinning convention that conflicts with this change. The change is consistent with the documented CI-pinning precedent (SHA-pinned v7 actions + 3.14 interpreter) and with the repository's overall stance that the review-quality gate should exercise the same interpreter as shipping CI. No prohibition on workflow comment style or quoted-scalar style exists in the standards file.

Linked Issue Fit

Issue PR 496 asks that the eval-harness workflow use the same pinned Python interpreter as ci.yaml and the same SHA-pinned action style, so the review-quality gate exercises the same interpreter as production CI. Verified against the issue's acceptance criteria:

  1. "pins the same Python version as ci.yaml (single source for the version, e.g. a repo-level default or matching literal)" — Met. python-version: '3.14' is now an exact matching literal to ci.yaml:24. While a repo-level default would be the ideal single source, the issue explicitly names "matching literal" as an acceptable form.
  2. "uses the same SHA-pinned action style" — Met. The current workflow file pins actions/checkout, actions/setup-python, and actions/upload-artifact to v7 SHAs (delivered by the in-flight Renovate PRs PR 491–493 that the issue anticipated); this PR completes the remaining interpreter gap.
  3. "The weekly scheduled eval run and CI both exercise the identical interpreter for the same pr_reviewer code" — Met. Both workflows now run Python 3.14.

No acceptance criterion is missed or misinterpreted; the PR's scope is exactly the interpreter alignment the issue identified as remaining.

@joryirving
joryirving merged commit bdeb2b0 into main Aug 12, 2026
2 checks passed
@joryirving
joryirving deleted the foreman/wl-misospace-pr-reviewer-action-496/issue-496 branch August 12, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Eval-harness workflow runs the review-quality gate on Python 3.12 with unpinned actions, while CI pins 3.14 by SHA

2 participants