Change array_caster in pybind11/stl.h to support value types that are not default-constructible.

[rwgk]

Description

Loosely related: pybind/pybind11#864

However, this PR is specific to array_caster; practically speaking, specific to std::array conversions.

This PR preserves the existing array_caster behavior as much as possible. The only difference is that it also works for value types that are not default-constructible. — Note that without this PR, the new test results in a compiler error.

This resolves failures when testing globally with PyCLIF-pybind11.

Suggested changelog entry:

[rainwoodman]

Is there a way to use e.g. emplace to construct the elements in place? then you can likely avoid the default constructible requirement. (not sure if the comment above means the requirement is a limitation or always satisfied)

[rwgk]

Yes, but I added this to the PR description after seeing your questions:

This PR preserves the existing array_caster behavior as much as possible. The only difference is that it also works for value types that are not default-constructible. — Note that without this PR, the new test results in a compiler error.

I also added a source code comment here:

        // Using `resize` to preserve the behavior exactly as it was before google/pywrapcc#30034

[rainwoodman]

Do you know typically from whom is src allocated? If so, might be helpful to mention that here for more context.

[rainwoodman]

Or is this a typical requirement for any take_ownership policy on casters with pointer type src?

[rwgk]

I also added a source code comment here:

// Intentionally preserving the behavior exactly as it was before google/pywrapcc#30034

Do you know typically from whom is src allocated?

No! :-)

I think this is one of the dark corners of pybind11, specifically the PYBIND11_TYPE_CASTER macro, which is used a lot. But the ecosystem has grown around it ... better don't touch.

[rainwoodman]

Do you allow mutation of members of the array? if so might want to add a test about whether mutating lst.arr affects m.NoDefaultCtorArray(4).arr.

[rwgk]

The array_caster copies (or moves) elements to/from a native Python list. The code below demonstrates the behavior. I verified that it works, then undid the change again, because that aspect of the array_caster behavior is tangential to adding the default ctor support.

def test_array_no_default_ctor():
    lst = m.NoDefaultCtorArray(3)
    assert [e.val for e in lst.arr] == [13, 23]
    lst4 = m.NoDefaultCtorArray(4)
    lst.arr = lst4.arr
    assert [e.val for e in lst.arr] == [14, 24]
    # Tangential to default ctor behavior:
    lst.arr = m.NoDefaultCtorArray(5).arr
    assert [e.val for e in lst.arr] == [15, 25]
    assert [e.val for e in lst4.arr] == [14, 24]

[rwgk]

EDIT: This was resolved by pybind/pybind11#5006

---

The two 🐍 3 • windows-latest • mingw32 CI failures are unrelated and also appear on pybind11 master:

https://github.com/pybind/pybind11/actions/runs/7514690075/job/20457837110
Installing additional packages through pacman...
137
C:\Windows\system32\cmd.exe /D /S /C D:\a_temp\setup-msys2\msys2.cmd -c "'pacman' '--noconfirm' '-S' '--needed' '--overwrite' '*' 'git' 'mingw-w64-i686-gcc' 'mingw-w64-i686-python-pip' 'mingw-w64-i686-python-numpy' 'mingw-w64-i686-python-scipy' 'mingw-w64-i686-cmake' 'mingw-w64-i686-make' 'mingw-w64-i686-python-pytest' 'mingw-w64-i686-eigen3' 'mingw-w64-i686-boost' 'mingw-w64-i686-catch'"
138
error: target not found: mingw-w64-i686-python-scipy
139
Error: The process 'C:\Windows\system32\cmd.exe' failed with exit code 1

Last to work: 2024-01-08T13:40:51.1780929Z

First broken: 2024-01-13T19:31:58.0408995Z

[rainwoodman]

Thanks for the changes!

[rwgk]

Thanks for the review!