Use cibuildwheel to replace multibuild #217

ffgan · 2025-10-19T12:31:52Z

see #211

below is Ci result:

All CI results show that everything went smoothly.

The detailed refactoring process can be viewed in the replies within the issue. To avoid making the PR too lengthy, the refactoring using cibuildwheel was only applied to Linux and macOS.

For Windows and Windows ARM, cibuildwheel has not been used yet. If you think it should be implemented using cibuildwheel in this PR as well, feel free to @ me directly, and I will submit a commit to implement it as soon as possible.

other info
Co-authored by: [email protected];

I updated the package version in pyproject.toml and made sure the first 3 numbers match git describe --tags --abbrev=8 in OpenBLAS at the OPENBLAS_COMMIT. If I did not update OPENBLAS_COMMIT, I incremented the wheel build number (i.e. 0.3.29.0.0 to 0.3.29.0.1)

Co-authored by: [email protected];

mattip

cibuildwheel does seem able to handle this, nice. I am not sure I am ready to transition away from multibuild, since I am the main project maintainer these days I do like to make sure it continues to work. Is there any clear advantage of cibuildwheel over multibuild for this type of package, which anyway invokes its own build/test scripts? The "wheel" is really only a repackaging of the OpenBLAS library with some pkgconfig additions.

A few general questions:

The macos-x86_64 build seems to be using a arm64 machine, which means that the tests are not run natively. Is there a way to use the macos-15-intel os instead of macos-latest for those builds?
travis is broken, the build is not running. We still need that for ppc64le and s390x (the s390x build usually fails, but we do have some users of ppc64le)

mattip · 2025-10-20T06:29:19Z

.github/workflows/posix.yml

+
+          - { os: ubuntu-24.04-arm, PLAT: aarch64, INTERFACE64: '0', MB_ML_VER: '_1_2', MB_ML_LIBC: musllinux}
+          - { os: ubuntu-24.04-arm, PLAT: aarch64, INTERFACE64: '1', MB_ML_VER: '_1_2', MB_ML_LIBC: musllinux}
+


Since we no longer use multibuild, the env variables should not have the MB_ prefix

To reduce the difficulty of review, I have tried to preserve the content of the original script as much as possible in this PR, without making excessive modifications. If you think modifications are needed, I can add a commit now.

I see it would be invasive in other places. Let's leave it for a clean-up PR.

mattip · 2025-10-20T06:46:12Z

tools/build_steps.sh

        # force the architecture
        arch -${PLAT} bash -s << EOF
-source ${ROOT_DIR}/gfortran-install/gfortran_utils.sh
+source tools/gfortran_utils.sh


In #216 I replaced this with an alias (and moved it inside the if $IS_OX), since gfortran is installed in the macos image.

alias gfortran=gfortran-15

Do you mean we should remove gfortran-install directly? I believe maintainers should be able to commit directly to this PR. If my understanding is wrong, please feel free to commit.

ffgan · 2025-10-20T08:02:39Z

cibuildwheel does seem able to handle this, nice. I am not sure I am ready to transition away from multibuild, since I am the main project maintainer these days I do like to make sure it continues to work. Is there any clear advantage of cibuildwheel over multibuild for this type of package, which anyway invokes its own build/test scripts? The "wheel" is really only a repackaging of the OpenBLAS library with some pkgconfig additions.

A few general questions:

The macos-x86_64 build seems to be using a arm64 machine, which means that the tests are not run natively. Is there a way to use the macos-15-intel os instead of macos-latest for those builds?

travis is broken, the build is not running. We still need that for ppc64le and s390x (the s390x build usually fails, but we do have some users of ppc64le)

Using cibuildwheel has the following advantages:

It can support more platforms. In fact, if this PR is successfully merged, I will soon propose support for riscv64.
cibuildwheel can better handle complex multi-platform scenarios, especially when building using containers on Linux. They have quite a lot of reliable containers that can be used directly.
cibuildwheel's build stages are reasonably split, allowing for good adjustments to the build logic. To reduce the difficulty of review, the current PR has not specifically adapted to cibuildwheel's stages, but tries to maintain the same logic as the openblas-libs build script.

Here are the answers to other questions:

macOS-x86-64 is not native, mainly because I tried building on macos-15-intel and it was quite slow. If you don't mind slow CI, macos-15-intel can be used for building.
Regarding the Travis issue, I'm now preparing to fix it. Once I've fixed it, I will immediately add commits to this PR.

rgommers · 2025-10-20T08:39:26Z

tools/build_steps.sh

+    # Report OS as given by uname
+    # Use any Python that comes to hand.
+    $(any_python) -c 'import platform; print(platform.uname()[0])'
+}


These functions don't seem used nor necessary

This code can be deleted, but as I mentioned before, I'm trying to avoid modifying the original script as much as possible to reduce the difficulty of the PR. If you think this code should be deleted directly, you can commit it—that's no problem.

rgommers · 2025-10-20T08:41:37Z

tools/build_steps.sh

+    #     -e MB_ML_VER=${manylinux} \
+    #     -e MB_ML_LIBC=${libc} \
+    #     -v $PWD:/io \
+    #     $docker_image /io/tools/docker_build_wrap.sh


I'd remove all of the commented out code from this PR, then it will be much clearer that the move to cibuildwheel is a significant reduction in complexity.

Sure, but please wait a moment. I'm trying to fix the Travis issue because I broke tools/build_wheel.sh. If I directly restore tools/build_wheel.sh, it will cause code redundancy issues (because I've split the code here into other places).

Now is fine. If you need to delete code, you can commit directly. Or if there's anything you need help with, feel free to point it out directly. I'm happy to assist you with the related content.

Co-authored by: [email protected];

ffgan · 2025-10-20T14:33:22Z

@mattip
Now Travis's status is consistent with the status of the most recent commit. Travis's second build failed, specifically PLAT=s390x INTERFACE64=1 MB_ML_VER=2014 failed. The specific reason is currently unknown due to lack of log.

mattip · 2025-10-20T16:18:13Z

Cool. Let's see what CI here thinks of this.

ffgan · 2025-10-20T23:37:33Z

The CI results show that only PLAT=s390x INTERFACE64=1 MB_ML_VER=2014 failed. Similarly, due to missing logs, debugging is temporarily not possible.

Additionally, it's important to note here that in this PR, the cibuildwheel refactoring does not include s390x/ppc64le and the Windows series. The reason is that I'm trying to minimize the burden of PR review as much as possible. If this PR gets merged, we can consider adding support for them using cibuildwheel in the future, but for now, let's keep the status quo.

ffgan added 2 commits October 19, 2025 16:51

Use cibuildwheel to replace multibuild

60f5496

Co-authored by: [email protected];

incremented the wheel build number

cce6ab5

Co-authored by: [email protected];

mattip reviewed Oct 20, 2025

View reviewed changes

rgommers reviewed Oct 20, 2025

View reviewed changes

ffgan added 6 commits October 20, 2025 17:24

fix travis error

e6c99f5

Co-authored by: [email protected];

fix travis error

ab8b10d

Co-authored by: [email protected];

fix travis error

4a70662

Co-authored by: [email protected];

fix travis error

fbb83f3

Co-authored by: [email protected];

try to debug travis

01b3d6d

Co-authored by: [email protected];

final fix travis error

6b406e9

Co-authored by: [email protected];


		- { os: ubuntu-24.04-arm, PLAT: aarch64, INTERFACE64: '0', MB_ML_VER: '_1_2', MB_ML_LIBC: musllinux}
		- { os: ubuntu-24.04-arm, PLAT: aarch64, INTERFACE64: '1', MB_ML_VER: '_1_2', MB_ML_LIBC: musllinux}

Use cibuildwheel to replace multibuild #217

Are you sure you want to change the base?

Use cibuildwheel to replace multibuild #217

Uh oh!

Conversation

ffgan commented Oct 19, 2025

see #211

Uh oh!

mattip left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ffgan commented Oct 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ffgan commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip commented Oct 20, 2025

Uh oh!

ffgan commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ffgan commented Oct 20, 2025 •

edited

Loading

ffgan commented Oct 20, 2025 •

edited

Loading