Skip to content

[ci] Fix PyPy venv caching issue #10392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 18, 2025

Conversation

cdce8p
Copy link
Member

@cdce8p cdce8p commented May 18, 2025

The tests job uses the python-version output from actions/setup-python as part of the cache key. That works well for cpython, however is broken for PyPy which doesn't include the Python version itself. Thus the workflow currently restores the first cached venv, e.g. pypy-3.11 on pypy-3.10, and thus no packages are actually available.

The current key prefix for PyPy (3.10 and 3.11): Linux-pypy7.3.19-venv-.... Better would be Linux-pypy3.10.16-7.3.19-venv-... / Linux-pypy3.11.11-7.3.19-venv-....

This isn't an issue on our backport branch since PyPy 3.9 was dropped recently and so the PyPy versions are actually different. Nevertheless it might be good to backport it as well.

IMO this is really an issue with the setup-python action. Opened an issue and PR upstream.

actions/setup-python#1109
actions/setup-python#1110

To unblock our workflow, I created a custom tag with just the fix.
cdce8p/setup-python@v5.6.0...v5.6.0-c1

@cdce8p cdce8p added this to the 3.3.8 milestone May 18, 2025
@cdce8p cdce8p added Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news 🔇 This change does not require a changelog entry backport maintenance/3.3.x labels May 18, 2025
Copy link

codecov bot commented May 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.90%. Comparing base (1257474) to head (03eaced).
Report is 10 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main   #10392   +/-   ##
=======================================
  Coverage   95.90%   95.90%           
=======================================
  Files         176      176           
  Lines       19122    19122           
=======================================
  Hits        18339    18339           
  Misses        783      783           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit 03eaced

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing catch, it's been bothering me since pypy 3.11 was added 😄 (should we wait for the fix un setup-python or should we add cdce8p to the trusted workflow ? It feels like it enlarge the surface of attack)

@cdce8p
Copy link
Member Author

cdce8p commented May 18, 2025

should we wait for the fix un setup-python or should we add cdce8p to the trusted workflow ? It feels like it enlarge the surface of attack

Does it increase the surface area though? It's my own fork and I have full access anyway. Yes, we could wait for the PR but the Github action staff can be slow to respond sometimes unfortunately. Might be worth it to unblock our CI in the meantime by adding cdce8p/setup-python@* as trusted workflow. (Already did so, otherwise the workflow wouldn't have run here probably.)

@Pierre-Sassoulas
Copy link
Member

No you're right if your account is compromised it's already game over

@Pierre-Sassoulas Pierre-Sassoulas merged commit 58813cd into pylint-dev:main May 18, 2025
41 checks passed
@pylint-dev-bot
Copy link

The backport to maintenance/3.3.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-maintenance/3.3.x maintenance/3.3.x
# Navigate to the new working tree
cd .worktrees/backport-maintenance/3.3.x
# Create a new branch
git switch --create backport-10392-to-maintenance/3.3.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 58813cd3627194395f6c0c893b217fb71cc425a3
# Push it to GitHub
git push --set-upstream origin backport-10392-to-maintenance/3.3.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-maintenance/3.3.x

Then, create a pull request where the base branch is maintenance/3.3.x and the compare/head branch is backport-10392-to-maintenance/3.3.x.

@cdce8p
Copy link
Member Author

cdce8p commented May 18, 2025

Let's not backport this one. The CI for the maintenance branch is fine without it.

@cdce8p cdce8p deleted the fix-pypy-cache branch May 18, 2025 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news 🔇 This change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants