Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a3400f1
Publish to PyPI via GitHub CI
EpicWink May 1, 2025
b7f3f7e
Don't cache pip in publish CI job
EpicWink May 3, 2025
759c36f
Improve language in noxfile comment
EpicWink May 3, 2025
70457e5
Use latest Python version available
EpicWink May 3, 2025
671d6ab
Switch PyPA action version comment to tag
EpicWink May 3, 2025
944e4ee
Split build out from release CI job
EpicWink May 3, 2025
cc9692b
Document approvals required for release
EpicWink May 4, 2025
d855d62
Link to version page in release deployment
EpicWink May 4, 2025
30cb9f2
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink May 4, 2025
6d6ba8e
Separate install from build in release CI job
EpicWink May 8, 2025
ed64a23
Link to CI job approval documentation
EpicWink May 8, 2025
b2a9652
Add punctuation to release process documentation
EpicWink May 8, 2025
94b6576
Check built distribution during release
EpicWink May 8, 2025
a0fc13e
Run twine check in strict mode during release
EpicWink May 8, 2025
f250545
Run twine-check in linting CI workflow
EpicWink May 9, 2025
9edb7a5
Remove redundant Python-setup step in release-build CI job
EpicWink May 9, 2025
ef36fbd
Switch to 'git switch' for switching branches
EpicWink Jul 29, 2025
c43d566
Merge branch 'main' into publish-ci-workflow
EpicWink Jul 29, 2025
5f36418
Switch to release 'published' event for CI trigger
EpicWink Jul 29, 2025
75cee2c
Revert "Run twine-check in linting CI workflow"
EpicWink Aug 6, 2025
2bc8119
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink Aug 6, 2025
d81fd82
Name checkout step
EpicWink Aug 7, 2025
100ed2f
Document install-only detection
EpicWink Aug 7, 2025
2eae318
Merge remote-tracking branch 'upstream/main' into publish-ci-workflow
EpicWink Sep 15, 2025
f5494b8
Use 'pypa/gh-action-pypi-publish' GitHub action v1.13
EpicWink Sep 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Publish

on:
release:
types: [published]
Comment on lines +4 to +5
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the last of my concerns. Having this inverse trigger can lead to weird sync issues.
For example, last week we were releasing pip-tools where this trigger is still used. And we ended up with a Git tag in repo and a GitHub Release existing for like 15-20 hours with no PyPI release, due to some approval challenges. Having a workflow_dispatch can prevent it when the sequence of actions performed is different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you want the workflow to instead be:

  • Push to main
  • Publish to PyPI (via manually triggering publish GitHub workflow)
  • Create and push Git tag
  • Create GitHub release

How do you prevent accidental triggering of publish GitHub workflow? A solution here is to prevent upload except when the wheel's version is of the form "X.Y.Z".

What if there's a problem during tag or release creation, where you have a version on PyPI with no associated tag? A solution here is to yank, but in my opinion this is a much worse outcome than the problems you mentioned, as these can be pinned by downstream users.

Copy link
Member

@webknjaz webknjaz Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you prevent accidental triggering of publish GitHub workflow? A solution here is to prevent upload except when the wheel's version is of the form "X.Y.Z".

Having environment: pypi on the publish job allows configuring the GitHub Environment (called pypi) protection rules. I normally add "required reviewers" in there (just adding oneself works). This makes it so when GH reaches the job, it pauses and does not proceed until whoever's listed clicked Approve on GH UI.

I like having an input with the version field in workflow_dispatch: https://github.com/cherrypy/cheroot/blob/ac9b6e5/.github/workflows/ci-cd.yml#L19-L28. And this event is only used for publishing so I don't tend to add checks for it. Though, I do have sanity-checks for the dist names. But yes, a check like you mention could be helpful too, I suppose.

What if there's a problem during tag or release creation, where you have a version on PyPI with no associated tag? A solution here is to yank, but in my opinion this is a much worse outcome than the problems you mentioned, as these can be pinned by downstream users.

To solve this, I separate the GH Release creation into a separate job, so that:

  1. it's possible to restart that job and it wouldn't attempt re-uploading the same thing to PyPI (which would otherwise crash)
  2. it's possible to fix whatever problem was there (semi-)manually (this could be wrapped as a Nox command for local invocation as a fallback)

And so with this in mind, in my workflows this doesn't constitute a problem. I'm very intentional about that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said above:

I don't have familiarity with using the workflow_dispatch GitHub workflows event, and don't feel comfortable switching to that workflow. If you want it, could you please make changes to this PR (or submit a new PR targeting this branch)?

I've never used workflow_dispatch for package release, so I'm not confident enough to write workflow definitions using it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EpicWink I can help. Should I send you a patch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@webknjaz yes please

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've encountered a significant issue with using workflow_dispatch to publish the workflow. In cryptography's publish workflow, the workflow_dispatch must be run from main, but the wheel can be built from any random branch/tag, as there's no hard linking from the ref the publish workflow is run on. In cryptography's case, the publish workflow uses a custom download-artifact action, which makes it harder to follow.

This break in the provenance chain, and I cannot rely on an artefact being generated by a GitHub workflow on a specific Git commit.

@webknjaz is there a way to better link the publish workflow to the build workflow?


env:
FORCE_COLOR: 1

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

# pipx is pre-installed in 'ubuntu' VMs

- name: Provision nox environment
run: pipx run nox --install-only

- name: Build distribution via nox
run: pipx run nox --no-install --error-on-missing-interpreters -s release_build

- name: Upload distribution
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # 4.6.2
with:
name: packages
path: dist/
if-no-files-found: error
compression-level: 0

publish:
environment:
name: pypi
url: https://pypi.org/project/packaging/${{ github.ref_name }}
permissions:
id-token: write

runs-on: ubuntu-latest

steps:
- name: Download distribution
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # 4.3.0
with:
name: packages
path: dist/

- name: Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with:
print-hash: true
7 changes: 6 additions & 1 deletion docs/development/release-process.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,13 @@ Release Process

$ nox -s release -- YY.N

You will need the password for your GPG key as well as an API token for PyPI.
This creates and pushes a new tag for the release.

#. Add a `release on GitHub <https://github.com/pypa/packaging/releases>`__.

This schedules a CI workflow which will build and publish the package to
PyPI. Publishing will wait for any `required approvals`_.

#. Notify the other project owners of the release.

.. note::
Expand All @@ -24,3 +27,5 @@ Release Process
- PyPI maintainer (or owner) access to ``packaging``
- push directly to the ``main`` branch on the source repository
- push tags directly to the source repository

.. _required approvals: https://docs.github.com/en/actions/managing-workflow-runs-and-deployments/managing-deployments/reviewing-deployments#approving-or-rejecting-a-job
88 changes: 68 additions & 20 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
import tempfile
import textwrap
import time
import webbrowser
from pathlib import Path

import nox
Expand Down Expand Up @@ -125,6 +124,9 @@ def release(session):
session.run("git", "add", str(changelog_file), external=True)
_bump(session, version=release_version, file=version_file, kind="release")

# Check the built distribution.
_build_and_check(session, release_version, remove=True)

# Tag the release commit.
# fmt: off
session.run(
Expand All @@ -143,11 +145,68 @@ def release(session):
next_version = f"{major}.{minor + 1}.dev0"
_bump(session, version=next_version, file=version_file, kind="development")

# Checkout the git tag.
session.run("git", "checkout", "-q", release_version, external=True)
# Push the commits and tag.
# NOTE: The following fails if pushing to the branch is not allowed. This can
# happen on GitHub, if the main branch is protected, there are required
# CI checks and "Include administrators" is enabled on the protection.
session.run("git", "push", "upstream", "main", release_version, external=True)


@nox.session
def release_build(session):
# Parse version from command-line arguments, if provided, otherwise get
# from Git tag.
try:
release_version = _get_version_from_arguments(session.posargs)
except ValueError as e:
if session.posargs:
session.error(f"Invalid arguments: {e}")

release_version = session.run(
"git", "describe", "--exact-match", silent=True, external=True
)
release_version = "" if release_version is None else release_version.strip()
session.debug(f"version: {release_version}")
checkout = False
else:
checkout = True

# Check state of working directory.
_check_working_directory_state(session)

# Ensure there are no uncommitted changes.
result = subprocess.run(
["git", "status", "--porcelain"], capture_output=True, encoding="utf-8"
)
if result.stdout:
print(result.stdout, end="", file=sys.stderr)
session.error("The working tree has uncommitted changes")

# Check out the Git tag, if provided.
if checkout:
session.run("git", "switch", "-q", release_version, external=True)

# Build the distribution.
_build_and_check(session, release_version)

# Get back out into main, if we checked out before.
if checkout:
session.run("git", "switch", "-q", "main", external=True)


def _build_and_check(session, release_version, remove=False):
package_name = "packaging"

session.install("build", "twine")

# Determine if we're in install-only mode. This works as `python --version`
# should always succeed when running `nox`, but in install-only mode
# `session.run(..., silent=True)` always immediately returns `None` instead
# of invoking the command and returning the command's output. See the
# documentation at:
# https://nox.thea.codes/en/stable/usage.html#skipping-everything-but-install-commands
install_only = session.run("python", "--version", silent=True) is None

# Build the distribution.
session.run("python", "-m", "build")

Expand All @@ -157,30 +216,19 @@ def release(session):
f"dist/{package_name}-{release_version}-py3-none-any.whl",
f"dist/{package_name}-{release_version}.tar.gz",
]
if files != expected:
if files != expected and not install_only:
diff_generator = difflib.context_diff(
expected, files, fromfile="expected", tofile="got", lineterm=""
)
diff = "\n".join(diff_generator)
session.error(f"Got the wrong files:\n{diff}")

# Get back out into main.
session.run("git", "checkout", "-q", "main", external=True)

# Check and upload distribution files.
session.run("twine", "check", *files)

# Push the commits and tag.
# NOTE: The following fails if pushing to the branch is not allowed. This can
# happen on GitHub, if the main branch is protected, there are required
# CI checks and "Include administrators" is enabled on the protection.
session.run("git", "push", "upstream", "main", release_version, external=True)

# Upload the distribution.
session.run("twine", "upload", *files)
# Check distribution files.
session.run("twine", "check", "--strict", *files)

# Open up the GitHub release page.
webbrowser.open("https://github.com/pypa/packaging/releases")
# Remove distribution files, if requested.
if remove and not install_only:
shutil.rmtree("dist", ignore_errors=True)


@nox.session
Expand Down
Loading