Skip to content

Commit ba22c64

Browse files
kramaranyakramaranya
authored
feat(ci): Add automated release CI job (#65)
* feat(ci): Add release CI job Signed-off-by: kramaranya <[email protected]> * feat(ci): Add GitHub Release to CI Signed-off-by: kramaranya <[email protected]> * Change kubeflow sdk PyPI package name Signed-off-by: kramaranya <[email protected]> * Add kubeflow sdk version verification to CI Signed-off-by: kramaranya <[email protected]> * Add proper changelog extraction Signed-off-by: kramaranya <[email protected]> * Allow to reuse test-python workflow Signed-off-by: kramaranya <[email protected]> * Add changelog generation script Signed-off-by: kramaranya <[email protected]> * Update uv.lock Signed-off-by: kramaranya <[email protected]> * Remove blank lines in changelog script Signed-off-by: kramaranya <[email protected]> * Add make release in Makefile Signed-off-by: kramaranya <[email protected]> * Refactor gen_changelog Signed-off-by: kramaranya <[email protected]> * Delete CHANGELOG.md Signed-off-by: kramaranya <[email protected]> * Add prepare-release CI Signed-off-by: kramaranya <[email protected]> * Allow manual trigger of release CI Signed-off-by: kramaranya <[email protected]> * Add release readme Signed-off-by: kramaranya <[email protected]> * Update chnagelog parser in CI Signed-off-by: kramaranya <[email protected]> * skip and check cherry pick for new release Signed-off-by: kramaranya <[email protected]> * add script to update the version Signed-off-by: kramaranya <[email protected]> * Update make release script Signed-off-by: kramaranya <[email protected]> * Use make test-python in release CI Signed-off-by: kramaranya <[email protected]> * Move prepare release CI to one a single workflow Signed-off-by: kramaranya <[email protected]> * Remove uv.sync from RELEASE.md Signed-off-by: kramaranya <[email protected]> * Upload artifacts to GitHub Release Signed-off-by: kramaranya <[email protected]> * Run release CI against release-* branch Signed-off-by: kramaranya <[email protected]> * Use X.Y.Z versioning in Kubeflow SDK Signed-off-by: kramaranya <[email protected]> * Use single workflow for release Signed-off-by: kramaranya <[email protected]> * Remove Release types from RELEASE.md Signed-off-by: kramaranya <[email protected]> * Add a note about older minor series patch release Signed-off-by: kramaranya <[email protected]> * Directly checkout release branch with ref input Signed-off-by: kramaranya <[email protected]> * Checkout release branch with ref input Signed-off-by: kramaranya <[email protected]> * Update extract changelog job Signed-off-by: kramaranya <[email protected]> * Update the name of github release Signed-off-by: kramaranya <[email protected]> * Remove full changelog line Signed-off-by: kramaranya <[email protected]> * Fix ruff issue Signed-off-by: kramaranya <[email protected]> * Don't use link for authors in changelog Signed-off-by: kramaranya <[email protected]> * Remove chnagelog for RCs Signed-off-by: kramaranya <[email protected]> --------- Signed-off-by: kramaranya <[email protected]>
1 parent 9c13d48 commit ba22c64

File tree

6 files changed

+649
-0
lines changed

6 files changed

+649
-0
lines changed

‎.github/workflows/release.yml‎

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- 'release-*'
8+
paths:
9+
- 'kubeflow/__init__.py'
10+
workflow_dispatch: {}
11+
12+
permissions:
13+
contents: write
14+
id-token: write
15+
16+
jobs:
17+
prepare:
18+
name: Prepare release branch
19+
if: ${{ !(github.event_name == 'push' && startsWith(github.ref, 'refs/heads/release-') && github.actor == 'github-actions[bot]') }}
20+
runs-on: ubuntu-latest
21+
outputs:
22+
version: ${{ steps.vars.outputs.version }}
23+
branch: ${{ steps.vars.outputs.branch }}
24+
is-prerelease: ${{ steps.vars.outputs.is-prerelease }}
25+
steps:
26+
- uses: actions/checkout@v4
27+
with:
28+
fetch-depth: 0
29+
30+
- name: Configure git user
31+
run: |
32+
git config user.name "github-actions[bot]"
33+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
34+
35+
- name: Check version and branch
36+
id: vars
37+
run: |
38+
VERSION=$(sed -n 's/^__version__ = "\(.*\)"/\1/p' kubeflow/__init__.py)
39+
MAJOR_MINOR=$(echo "$VERSION" | cut -d. -f1,2)
40+
BRANCH=release-$MAJOR_MINOR
41+
42+
echo "version=$VERSION" >> $GITHUB_OUTPUT
43+
echo "branch=$BRANCH" >> $GITHUB_OUTPUT
44+
45+
if [[ "$VERSION" =~ rc[0-9]+$ ]]; then
46+
echo "is-prerelease=true" >> $GITHUB_OUTPUT
47+
else
48+
echo "is-prerelease=false" >> $GITHUB_OUTPUT
49+
fi
50+
51+
- name: Ensure release branch exists and contains version bump
52+
run: |
53+
set -euo pipefail
54+
VERSION="${{ steps.vars.outputs.version }}"
55+
BRANCH="${{ steps.vars.outputs.branch }}"
56+
MAIN_SHA="${{ github.sha }}"
57+
58+
if [[ "${GITHUB_REF_NAME}" == "$BRANCH" ]]; then
59+
echo "Triggered on $BRANCH. Skipping cherry-pick from main."
60+
exit 0
61+
fi
62+
63+
if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then
64+
echo "Using existing branch: $BRANCH"
65+
git fetch origin "$BRANCH":"$BRANCH"
66+
git checkout "$BRANCH"
67+
if git merge-base --is-ancestor "$MAIN_SHA" "$BRANCH"; then
68+
echo "Commit $MAIN_SHA already present in $BRANCH. Skipping cherry-pick."
69+
else
70+
if ! git cherry-pick -x "$MAIN_SHA"; then
71+
echo "Cherry-pick failed. Please resolve manually on $BRANCH." >&2
72+
exit 1
73+
fi
74+
fi
75+
else
76+
echo "Creating new branch: $BRANCH from main@$MAIN_SHA"
77+
git checkout -B "$BRANCH" "$MAIN_SHA"
78+
fi
79+
git push origin "$BRANCH"
80+
81+
build:
82+
name: Build package
83+
needs: [prepare]
84+
runs-on: ubuntu-latest
85+
steps:
86+
- uses: actions/checkout@v4
87+
with:
88+
fetch-depth: 0
89+
ref: ${{ needs.prepare.outputs.branch }}
90+
91+
- name: Set up Python
92+
uses: actions/setup-python@v5
93+
with:
94+
python-version: '3.11'
95+
96+
- name: Setup build environment
97+
run: |
98+
make verify
99+
100+
- name: Run unit tests
101+
run: |
102+
make test-python
103+
104+
- name: Verify version
105+
run: |
106+
TAG_VERSION="${{ needs.prepare.outputs.version }}"
107+
CODE_VERSION="$(python -c "import kubeflow; print(kubeflow.__version__)")"
108+
echo "Tag version: $TAG_VERSION"
109+
echo "Code version: $CODE_VERSION"
110+
if [[ "$TAG_VERSION" != "$CODE_VERSION" ]]; then
111+
echo "Version mismatch"; exit 1; fi
112+
echo "Version verified: $TAG_VERSION"
113+
114+
- name: Build and validate package
115+
run: |
116+
uv build
117+
uvx twine check dist/*
118+
119+
- name: Upload build artifacts
120+
uses: actions/upload-artifact@v4
121+
with:
122+
name: dist-${{ needs.prepare.outputs.version }}
123+
path: dist/
124+
125+
create-tag:
126+
name: Create and push tag
127+
needs: [prepare, build]
128+
runs-on: ubuntu-latest
129+
steps:
130+
- uses: actions/checkout@v4
131+
with:
132+
fetch-depth: 0
133+
ref: ${{ needs.prepare.outputs.branch }}
134+
- name: Create tag
135+
run: |
136+
VERSION="${{ needs.prepare.outputs.version }}"
137+
if git ls-remote --tags origin "$VERSION" | grep -q "refs/tags/$VERSION"; then
138+
echo "Tag $VERSION already exists. Skipping"; exit 0; fi
139+
git tag "$VERSION"
140+
git push origin "$VERSION"
141+
142+
publish-pypi:
143+
name: Publish to PyPI
144+
needs: [prepare, build, create-tag]
145+
runs-on: ubuntu-latest
146+
environment:
147+
name: release
148+
url: https://pypi.org/project/kubeflow/
149+
steps:
150+
- name: Download build artifacts
151+
uses: actions/download-artifact@v4
152+
with:
153+
name: dist-${{ needs.prepare.outputs.version }}
154+
path: dist/
155+
- name: Publish to PyPI
156+
uses: pypa/gh-action-pypi-publish@release/v1
157+
with:
158+
verbose: true
159+
160+
github-release:
161+
name: Create GitHub Release
162+
needs: [prepare, build, create-tag, publish-pypi]
163+
runs-on: ubuntu-latest
164+
environment:
165+
name: release
166+
url: https://github.com/kubeflow/sdk/releases
167+
steps:
168+
- uses: actions/checkout@v4
169+
with:
170+
fetch-depth: 0
171+
ref: ${{ needs.prepare.outputs.branch }}
172+
- name: Download build artifacts
173+
uses: actions/download-artifact@v4
174+
with:
175+
name: dist-${{ needs.prepare.outputs.version }}
176+
path: dist/
177+
- name: Extract changelog
178+
if: needs.prepare.outputs.is-prerelease != 'true'
179+
id: changelog
180+
run: |
181+
VERSION="${{ needs.prepare.outputs.version }}"
182+
MAJOR_MINOR=$(echo "$VERSION" | cut -d. -f1,2)
183+
CHANGELOG_FILE="CHANGELOG/CHANGELOG-${MAJOR_MINOR}.md"
184+
set -euo pipefail
185+
[[ -f "$CHANGELOG_FILE" ]] || { echo "ERROR: $CHANGELOG_FILE not found" >&2; exit 1; }
186+
HEADER_REGEX="^# \\[${VERSION//./\\.}\\]"
187+
SECTION=$(sed -n "/$HEADER_REGEX/,\$p" "$CHANGELOG_FILE" | tail -n +2)
188+
[[ -n "$SECTION" ]] || { echo "ERROR: No changelog section for $VERSION in $CHANGELOG_FILE" >&2; exit 1; }
189+
NEXT_VERSION=$(echo "$SECTION" | grep -m1 "^# \\[[0-9]" || true)
190+
if [[ -n "$NEXT_VERSION" ]]; then
191+
CHANGELOG=$(echo "$SECTION" | sed -n "1,/^# \\[[0-9]/p" | sed '1d;$d')
192+
else
193+
CHANGELOG=$(echo "$SECTION" | sed '1d')
194+
fi
195+
[[ -n "$CHANGELOG" ]] || { echo "ERROR: Empty changelog body for $VERSION in $CHANGELOG_FILE" >&2; exit 1; }
196+
{
197+
echo "changelog<<EOF"
198+
echo "$CHANGELOG"
199+
echo "EOF"
200+
} >> $GITHUB_OUTPUT
201+
- name: Create GitHub Release
202+
uses: softprops/action-gh-release@v1
203+
with:
204+
tag_name: ${{ needs.prepare.outputs.version }}
205+
name: ${{ needs.prepare.outputs.version }}
206+
body: ${{ steps.changelog.outputs.changelog }}
207+
draft: false
208+
prerelease: ${{ needs.prepare.outputs.is-prerelease == 'true' }}
209+
generate_release_notes: false
210+
files: |
211+
dist/*

‎Makefile‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,14 @@ uv-venv:
6565
echo "uv virtual environment already exists in $(VENV_DIR)."; \
6666
fi
6767

68+
.PHONY: release
69+
release: install-dev
70+
@if [ -z "$(VERSION)" ]; then echo "Usage: make release VERSION=0.1.0"; exit 1; fi
71+
@V_NO_V=$(VERSION); \
72+
sed -i.bak "s/^__version__ = \".*\"/__version__ = \"$$V_NO_V\"/" kubeflow/__init__.py && \
73+
rm -f kubeflow/__init__.py.bak
74+
@uv run python scripts/gen-changelog.py --token=$${GITHUB_TOKEN} --version=$(VERSION)
75+
6876
# make test-python will produce html coverage by default. Run with `make test-python report=xml` to produce xml report.
6977
.PHONY: test-python
7078
test-python: uv-venv

‎RELEASE.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Releasing the Kubeflow SDK
2+
3+
## Prerequisites
4+
5+
- [Write](https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization#permission-levels-for-repositories-owned-by-an-organization)
6+
permission for the Kubeflow SDK repository.
7+
8+
- Create a [GitHub Token](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) and set it as `GITHUB_TOKEN` environment variable.
9+
10+
## Versioning Policy
11+
12+
Kubeflow SDK version format follows Python's [PEP 440](https://peps.python.org/pep-0440/).
13+
Kubeflow SDK versions are in the format of `X.Y.Z`, where `X` is the major version, `Y` is
14+
the minor version, and `Z` is the patch version.
15+
The patch version contains only bug fixes.
16+
17+
Additionally, Kubeflow SDK does pre-releases in this format: `X.Y.ZrcN` where `N` is a number
18+
of the `Nth` release candidate (RC) before an upcoming public release named `X.Y.Z`.
19+
20+
## Release Branches and Tags
21+
22+
Kubeflow SDK releases are tagged with tags like `X.Y.Z`, for example `0.1.0`.
23+
24+
Release branches are in the format of `release-X.Y`, where `X.Y` stands for
25+
the minor release.
26+
27+
`X.Y.Z` releases are released from the `release-X.Y` branch. For example,
28+
`0.1.0` release should be on `release-0.1` branch.
29+
30+
If you want to push changes to the `release-X.Y` release branch, you have to
31+
cherry pick your changes from the `main` branch and submit a PR.
32+
33+
## Changelog Structure
34+
35+
Kubeflow SDK uses a directory-based changelog structure under `CHANGELOG/`:
36+
37+
```
38+
CHANGELOG/
39+
├── CHANGELOG-0.1.md # All 0.1.x releases
40+
├── CHANGELOG-0.2.md # All 0.2.x releases
41+
└── CHANGELOG-0.3.md # All 0.3.x releases
42+
```
43+
44+
Each file contains releases for that minor series, with the most recent releases at the top.
45+
46+
## Release Process
47+
48+
### Automated Release Workflow
49+
50+
The Kubeflow SDK uses an automated release process with GitHub Actions:
51+
52+
1. **Local Preparation**: Update version and generate changelog locally
53+
2. **Automated CI**: GitHub Actions handles branch creation, tagging, building, and publishing
54+
3. **Manual Approvals**: PyPI and GitHub releases require manual approval
55+
56+
### Step-by-Step Release Process
57+
58+
#### 1. Update Version and Changelog
59+
60+
1. Generate version and changelog locally (this will sync dependencies automatically):
61+
62+
```sh
63+
export GITHUB_TOKEN=<your_github_token>
64+
make release VERSION=X.Y.Z
65+
```
66+
67+
This updates:
68+
- `kubeflow/__init__.py` with `__version__ = "X.Y.Z"`
69+
- `CHANGELOG/CHANGELOG-X.Y.md` with a new top entry `# [X.Y.Z] (YYYY-MM-DD)`
70+
71+
2. Open a PR:
72+
- Review `kubeflow/__init__.py` and `CHANGELOG/CHANGELOG-X.Y.md`
73+
- **For latest minor series**: Open a PR to `main` and get it reviewed and merged
74+
- **For older minor series patch (e.g. 0.1.1 when main is at 0.2.x)**: Open a PR to the corresponding `release-X.Y` branch
75+
76+
#### 2. Automated Release Process
77+
78+
The `Release` GitHub Action automatically:
79+
80+
1. **Prepare**: Detects the version change in `kubeflow/__init__.py` and creates or updates the `release-X.Y` branch
81+
2. **Build**: Runs tests and builds the package on the release branch
82+
3. **Tag**: Creates and pushes the release tag
83+
4. **Publish**: Publishes to PyPI (requires manual approval)
84+
5. **Release**: Creates GitHub Release (requires manual approval)
85+
86+
**Verification**: Confirm the release branch and tag were created!
87+
88+
#### 3. Manual Approvals
89+
90+
1. **PyPI Publishing**: Go to [GitHub Actions](https://github.com/kubeflow/sdk/actions) → `Release` workflow → Approve "Publish to PyPI"
91+
92+
2. **GitHub Release**: After PyPI approval → Approve "Create GitHub Release"
93+
94+
#### 4. Final Verification
95+
96+
1. Verify the release on [PyPI](https://pypi.org/project/kubeflow/)
97+
2. Verify the release on [GitHub Releases](https://github.com/kubeflow/sdk/releases)
98+
3. Test installation: `pip install kubeflow==X.Y.Z`
99+
100+
101+
## Announcement
102+
103+
**Announce**: Post the announcement for the new Kubeflow SDK release in:
104+
- [#kubeflow-ml-experience](https://www.kubeflow.org/docs/about/community/#slack-channels) Slack channel
105+
- [kubeflow-discuss](https://www.kubeflow.org/docs/about/community/#kubeflow-mailing-list) mailing list

‎pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ dev = [
4040
"kubeflow_trainer_api@git+https://github.com/kubeflow/trainer.git@master#subdirectory=api/python_api",
4141
"ruff>=0.12.2",
4242
"pre-commit>=4.2.0",
43+
"PyGithub>=2.7.0",
4344
]
4445

4546
[project.urls]

0 commit comments

Comments
 (0)