-
Notifications
You must be signed in to change notification settings - Fork 273
Publish to PyPI via GitHub CI #893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
EpicWink
wants to merge
25
commits into
pypa:main
Choose a base branch
from
EpicWink:publish-ci-workflow
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+127
−21
Open
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
a3400f1
Publish to PyPI via GitHub CI
EpicWink b7f3f7e
Don't cache pip in publish CI job
EpicWink 759c36f
Improve language in noxfile comment
EpicWink 70457e5
Use latest Python version available
EpicWink 671d6ab
Switch PyPA action version comment to tag
EpicWink 944e4ee
Split build out from release CI job
EpicWink cc9692b
Document approvals required for release
EpicWink d855d62
Link to version page in release deployment
EpicWink 30cb9f2
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink 6d6ba8e
Separate install from build in release CI job
EpicWink ed64a23
Link to CI job approval documentation
EpicWink b2a9652
Add punctuation to release process documentation
EpicWink 94b6576
Check built distribution during release
EpicWink a0fc13e
Run twine check in strict mode during release
EpicWink f250545
Run twine-check in linting CI workflow
EpicWink 9edb7a5
Remove redundant Python-setup step in release-build CI job
EpicWink ef36fbd
Switch to 'git switch' for switching branches
EpicWink c43d566
Merge branch 'main' into publish-ci-workflow
EpicWink 5f36418
Switch to release 'published' event for CI trigger
EpicWink 75cee2c
Revert "Run twine-check in linting CI workflow"
EpicWink 2bc8119
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink d81fd82
Name checkout step
EpicWink 100ed2f
Document install-only detection
EpicWink 2eae318
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink f5494b8
Use 'pypa/gh-action-pypi-publish' GitHub action v1.13
EpicWink File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| name: Publish | ||
|
|
||
| on: | ||
| release: | ||
| types: [published] | ||
|
|
||
| env: | ||
| FORCE_COLOR: 1 | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
|
|
||
| # pipx is pre-installed in 'ubuntu' VMs | ||
|
|
||
| - name: Provision nox environment | ||
| run: pipx run nox --install-only | ||
|
|
||
| - name: Build distribution via nox | ||
| run: pipx run nox --no-install --error-on-missing-interpreters -s release_build | ||
|
|
||
| - name: Upload distribution | ||
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # 4.6.2 | ||
| with: | ||
| name: packages | ||
| path: dist/ | ||
| if-no-files-found: error | ||
| compression-level: 0 | ||
EpicWink marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| publish: | ||
| environment: | ||
| name: pypi | ||
| url: https://pypi.org/project/packaging/${{ github.ref_name }} | ||
| permissions: | ||
| id-token: write | ||
EpicWink marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Download distribution | ||
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # 4.3.0 | ||
| with: | ||
| name: packages | ||
| path: dist/ | ||
|
|
||
| - name: Publish distribution to PyPI | ||
| uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 | ||
| with: | ||
| print-hash: true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the last of my concerns. Having this inverse trigger can lead to weird sync issues.
For example, last week we were releasing
pip-toolswhere this trigger is still used. And we ended up with a Git tag in repo and a GitHub Release existing for like 15-20 hours with no PyPI release, due to some approval challenges. Having aworkflow_dispatchcan prevent it when the sequence of actions performed is different.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume you want the workflow to instead be:
mainHow do you prevent accidental triggering of publish GitHub workflow? A solution here is to prevent upload except when the wheel's version is of the form "X.Y.Z".
What if there's a problem during tag or release creation, where you have a version on PyPI with no associated tag? A solution here is to yank, but in my opinion this is a much worse outcome than the problems you mentioned, as these can be pinned by downstream users.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having
environment: pypion the publish job allows configuring the GitHub Environment (calledpypi) protection rules. I normally add "required reviewers" in there (just adding oneself works). This makes it so when GH reaches the job, it pauses and does not proceed until whoever's listed clickedApproveon GH UI.I like having an input with the version field in
workflow_dispatch: https://github.com/cherrypy/cheroot/blob/ac9b6e5/.github/workflows/ci-cd.yml#L19-L28. And this event is only used for publishing so I don't tend to add checks for it. Though, I do have sanity-checks for the dist names. But yes, a check like you mention could be helpful too, I suppose.To solve this, I separate the GH Release creation into a separate job, so that:
And so with this in mind, in my workflows this doesn't constitute a problem. I'm very intentional about that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I said above:
I've never used
workflow_dispatchfor package release, so I'm not confident enough to write workflow definitions using it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@EpicWink I can help. Should I send you a patch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@webknjaz yes please
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've encountered a significant issue with using
workflow_dispatchto publish the workflow. In cryptography's publish workflow, theworkflow_dispatchmust be run frommain, but the wheel can be built from any random branch/tag, as there's no hard linking from the ref the publish workflow is run on. Incryptography's case, the publish workflow uses a custom download-artifact action, which makes it harder to follow.This break in the provenance chain, and I cannot rely on an artefact being generated by a GitHub workflow on a specific Git commit.
@webknjaz is there a way to better link the publish workflow to the build workflow?