Skip to content

⚡️ Improve Python CI performance - #1944

Merged
burgholzer merged 2 commits into
mainfrom
agent/ci-python-performance
Jul 29, 2026
Merged

⚡️ Improve Python CI performance#1944
burgholzer merged 2 commits into
mainfrom
agent/ci-python-performance

Conversation

@burgholzer

@burgholzer burgholzer commented Jul 25, 2026

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Summary

  • Update all reusable workflow pins to the immutable v2.2.1 release commit.
  • Share one scikit-build-core build directory across Python test sessions.
  • Configure coverage.py's sysmon core and parallel data collection centrally.
  • Require pytest-cov 7.1 for the coverage configuration.

The shared build directory lets the minimum- and regular-resolution nox sessions reuse the same CMake and Ninja state. On Windows, reusable workflows v2.2.1 initialize native MSVC and use Ninja so the existing job-local sccache installation is active.

Evidence

The final test-only experiment passed on macOS and Windows in run 30434315809. The full Core CI run for the same source revision also passed on every supported runner in run 30434310296.

The strongest reuse was on Windows: after the minimum-resolution builds, all five regular-resolution sessions completed in 47 seconds and sccache reported a 35.93% hit rate in run 30404816970.

Validation

  • uv lock --check
  • Full Core lint session
  • macOS and Windows performance experiment
  • Full Core CI matrix
  • No mutable workflow references or benchmark-only workflow remain
  • I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

@burgholzer burgholzer changed the title 🧪 Test the optimized reusable Python workflow 🧪 Benchmark optimized reusable Python workflows Jul 26, 2026
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Comment thread pyproject.toml Outdated
@burgholzer
burgholzer force-pushed the agent/ci-python-performance branch from 2fc16a5 to 11dfa42 Compare July 29, 2026 08:02
@burgholzer burgholzer changed the title 🧪 Benchmark optimized reusable Python workflows 🧪 Benchmark reusable Python test builds Jul 29, 2026
@burgholzer
burgholzer force-pushed the agent/ci-python-performance branch from 11dfa42 to ad51c1f Compare July 29, 2026 08:05
Exercise the reusable test workflow with a shared native build tree and retain the parallel coverage integration.

Co-authored-by: Daniel Haag <121057143+denialhaag@users.noreply.github.com>
Assisted-by: GPT-5.6 via Codex
@burgholzer
burgholzer force-pushed the agent/ci-python-performance branch from ad51c1f to 976bd3f Compare July 29, 2026 08:08
Replace the benchmark-only feature reference with the released workflow and update all reusable workflow pins consistently.

Assisted-by: GPT-5.6 via Codex
@burgholzer burgholzer changed the title 🧪 Benchmark reusable Python test builds ⚡️ Improve Python CI performance Jul 29, 2026
@burgholzer
burgholzer marked this pull request as ready for review July 29, 2026 12:28
@burgholzer burgholzer added the continuous integration Anything related to the CI setup label Jul 29, 2026
@burgholzer burgholzer self-assigned this Jul 29, 2026
@burgholzer burgholzer added the python Anything related to Python code label Jul 29, 2026
@burgholzer
burgholzer enabled auto-merge (squash) July 29, 2026 12:33
@burgholzer
burgholzer merged commit 3e3c7ba into main Jul 29, 2026
42 checks passed
@burgholzer
burgholzer deleted the agent/ci-python-performance branch July 29, 2026 13:31
burgholzer added a commit that referenced this pull request Jul 30, 2026
🤖 *AI text below* 🤖

## Description

Backport selected changes from `main` to the maintained `v3.x` branch in
preparation for MQT Core 3.8.0.

The commits are kept separate by upstream change wherever practical. The
series includes:

- configurable QDMI device discovery and stable-ID registration from
#1912
- binary-safe FoMaC/QDMI program handling from #1957
- CMake subproject composability and individually selectable bundled
QDMI devices from the current #1965 head
(`cdc740398e472c19b585cf6dbd7d41de053d84a6`)
- optional QIR support for the DDSIM QDMI device and the associated QIR
runtime improvements
- native RCCX support and the bottom-up DD construction path
- Python CI performance improvements from #1944
- template synchronization through #1964, agent/contributor guide
updates, dependency and pre-commit updates, and related maintenance

## v3 compatibility

This backport deliberately preserves the v3 architecture:

- LLVM/MLIR remains optional and continues to use LLVM/MLIR 21.
- QIR support is available behind the existing MLIR-dependent feature
flags.
- The mandatory LLVM/MLIR 22 transition from #1953 and
compiler-collection changes that depend on the v4 MLIR/QCO redesign are
not included.
- #1965 is adapted to retain optional MLIR and the v3 documentation
roles while preserving its CMake and QDMI behavior.

## Validation

- configured and built the complete non-MLIR library
- configured the MLIR build against LLVM/MLIR 21.1.8
- built and ran focused DD, FoMaC, QDMI driver/registry/device, and QIR
tests
- verified 435 #1965-affected C++ test cases, with two expected device
capability skips
- configured MQT Core as an embedded CMake subproject with all bundled
devices disabled by default
- configured an embedded build with only the DDSIM QDMI device enabled
- built and ran the device-free registry suite: 11 tests passed
- focused Python FoMaC suite: 181 passed, two expected non-MLIR skips
- regenerated Python stubs without a diff
- complete lint, lockfile validation, commit-signature audit, and `git
diff --check` passed

The pre-existing v3 test configuration still requires `llc` even when
MLIR is disabled. The non-MLIR library build itself succeeds; MLIR/QIR
validation used LLVM/MLIR 21.

## Checklist

- [x] The pull request only contains commits that are focused and
relevant to this change.
- [x] I have added appropriate tests that cover the new/changed
functionality.
- [x] I have updated the documentation to reflect these changes.
- [x] I have added entries to the changelog for any noteworthy
additions, changes, fixes, or removals.
- [x] I have added migration instructions to the upgrade guide (if
needed).
- [x] The changes follow the project's style guidelines and introduce no
new warnings.
- [x] The changes are fully tested and pass the CI checks.
- [x] I have reviewed my own code changes.

**If PR contains AI-assisted content:**

- [x] Any agent that created, edited, or submitted GitHub content was
explicitly authorized for that scope, as required by our AI Usage
Guidelines.
- [x] Every agent-authored or agent-edited public text body begins with
the visible disclosure `🤖 *AI text below* 🤖` (titles are exempt).
- [x] I have disclosed AI assistance in the PR description.
- [x] I confirm that I have personally reviewed and understood all
AI-generated content, and accept full responsibility for it.

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Lukas Burgholzer <burgholzer@me.com>
Signed-off-by: Roberto Turrado Camblor <rturrado@gmail.com>
Signed-off-by: rturrado <rturrado@gmail.com>
Co-authored-by: mqt-app[bot] <219534693+mqt-app[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Haag <121057143+denialhaag@users.noreply.github.com>
Co-authored-by: simon1hofmann <119581649+simon1hofmann@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Roberto Turrado Camblor <rturrado@gmail.com>
Co-authored-by: Yannick Stade <100073938+ystade@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

continuous integration Anything related to the CI setup python Anything related to Python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant