NumPy does not install on Python 3.12.0b1

[bersbersbers]

Describe the issue:

Related:
pypa/pip#11501 (comment)
pypa/setuptools#3935

https://github.com/numpy/numpy/blob/cc0abd768575d7f9e862de0b4912af27f6e9690d/pyproject.toml#LL7C6-L7C16

Reproduce the code example:

pip install numpy

Error message:

pip install numpy
Collecting numpy
  Using cached numpy-1.24.3.tar.gz (10.9 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "C:\Users\bers\.pyenv-win-venv\envs\project_3.12\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\bers\.pyenv-win-venv\envs\project_3.12\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv-win-venv\envs\project_3.12\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv-win-venv\envs\project_3.12\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b1\Lib\importlib\__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1216, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1237, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 841, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 994, in exec_module
        File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
        File "C:\Users\bers\AppData\Local\Temp\pip-build-env-8dt1uf4c\overlay\Lib\site-packages\setuptools\__init__.py", line 16, in <module>
          import setuptools.version
        File "C:\Users\bers\AppData\Local\Temp\pip-build-env-8dt1uf4c\overlay\Lib\site-packages\setuptools\version.py", line 1, in <module>
          import pkg_resources
        File "C:\Users\bers\AppData\Local\Temp\pip-build-env-8dt1uf4c\overlay\Lib\site-packages\pkg_resources\__init__.py", line 2172, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Runtime information:

Cannot install numpy :)
Python 3.12.0b1 on Windows 10 21H2

Context for the issue:

No response

[mattip]

We deliberately pin setuptools to work around other problems with numpy.distutils. The plan for python 3.12 is to move to the meson build system, and to deprecate use of setuptools. You can try out the meson build using python3.12 -m pip install -r build_requirements.txt; python3.12 -m spin build.

[bersbersbers]

Thanks, understood!

I just came across https://numpy.org/devdocs/reference/distutils_status_migration.html, which reads

It may be beneficial to wait with migrating until there are examples from other projects to follow

I agree in principle, but I feel that this approach may deadlock if many other projects depend on numpy :) for example, matplotlib does not install on 3.12.0b1 because it depends on numpy at build time, and then that dependency is carried further to all projects depending on matplotlib.

[mattip]

That advice could use an update, especially before the 1.25 release freezes the documentation. SciPy has migrated to meson-python, and NumPy should migrate soon. Rather than trying to get setuptools to work with python3.12, we should start testing the CPython 3.12b1 with meson.

[charris]

That advice could use an update, especially before the 1.25 release freezes the documentation.

Could you open an issue for that? I am going to release 1.25.0rc1 without it, but there will still be time before the final.

[mattip]

matplotlib does not install on 3.12.0b1 because it depends on numpy at build time,

Maybe we should be uploading pre-release wheels to https://anaconda.org/scientific-python-nightly-wheels/numpy. It seems cibuildwheel has a option for this

[rgommers]

I think now that 1.25.x has branched, we can make the switch to Meson by default in main. That will make pip install . & co work with Python 3.12 beta's (module any crashes due to incompatibilities between CPython, NumPy and Cython of course - and we know we have one of those right now: gh-23766). WDYT?

Re uploading a nightly: let's consider it once NumPy actually passes its test suite with a CPython beta? It seems too early now, we don't even run it in our own CI yet.

[mattip]

We need SIMD testing to work to make the transition.

[rgommers]

We need SIMD testing to work to make the transition.

That can still be done with python setup.py install, as is already done in armv7_simd_test.