Skip to content

WIP: ENH: implement support for build-details.json (PEP 739) and cross compilation #779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rgommers
Copy link
Contributor

@rgommers rgommers commented Jul 16, 2025

This is very much not ready yet, but I'm opening a WIP PR now to get early feedback, since there's more than one way this can be implemented.

Depends on mesonbuild/meson#14657 (Cc @mgorny)
Closes gh-321

Example of a cross compilation CI job for SciPy, which successfully uses a cross file and a build-details.json file to cross compile from x86-64 to aarch64 and then in a second job installs and tests the cross-compiled wheel:

The meson-python parts of cross compilation that need fixing are:

  1. The wheel filename (interpreter, abi and platform tags)
  2. The platform tag in the WHEEL metadata file

The changes here do both of those things.

A few things I'd like to get input on:

  • I chose to use both is_cross and build_details in _WheelBuilder, for clarity and so that cross compilation without use of a build-details.json file can also be supported (parsing the cross file can get the platform right at least, but not the interpreter). We could also decide to not worry about that case and only support supplying both files for a cross build.
  • Test strategy:
    • Cross compile a single package, or multiple/all?
    • Do we do just Linux x86-64 -> aarch64 and vice versa as tests where we can validate at runtime because easy with GHA runners and toolchain setup, or anything else?
    • Conda-forge is easiest to set up (for me at least), but we can also do another distro.
    • Use minimum supported Python and Python 3.14? For the former we need to generate a build-details.json, for the latter it's shipped as part of CPython.
  • (minor) Static typing: build_details is hard to type. Use build_details: Optional[Dict[str, str | Dict[str, Any]]] or simply Dict[str, Any]?

@rgommers rgommers added the enhancement New feature or request label Jul 16, 2025
@rgommers rgommers marked this pull request as draft July 16, 2025 19:15
@dnicolodi
Copy link
Member

I haven't read PEP 739 recently, but I was under the impression that PEP 739 will also be used for building native wheels, not only for cross compilation, or in other words that given a Python interpreter it should be possible to load a build-details.json describing the interpreter, sparing build tools from doing the kind of messy interpreter introspection meson-python needs to implement in _tag.py. However, as implemented here, the support for PEP 739 is enabled only in the case of cross compilation. Did I misunderstand the intent of PEP 739?

@rgommers
Copy link
Contributor Author

Good point. Yes, there isn't any reason that build-details.json cannot be used for a native build. So we should support that as well. That said, there isn't a benefit from the user passing in such a file explicitly; we can start reading it automatically for Python >=3.14, but I'm a bit hesitant to do that right away since it's so new. We can start doing that at any point in the future if it helps simplify our internal code, since it's a backwards-compatible change.

So for now in this PR we can distinguish three cases, with the user passing:

  1. Only a cross file (or multiple cross files)
  2. Only a build-details.json file
  3. Both a cross file and a build-details.json file

I focused on (3) first and have a TODO in here already to add (1); I should add (2) as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support cross compiling
2 participants