[BUG]: Windows Python with Debug Libraries Installed causes linker error

[WhoBrokeTheBuild]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

When installing python for windows, you have the option of installing the debug binaries as well.

If you then find_package(Python) followed by find_package(pybind11) it will cause the following linker error:

[build] LINK : fatal error LNK1104: cannot open file 'python310.lib'

If you then search the link command line, I cannot find any mention of python310.lib, only "C:/Program Files/Python310/libs/python310_d.lib".

The find_package(Python) finds C:/Program Files/Python310/libs/python310_d.lib
The find_package(pybind11) finds C:/Program Files/Python310/libs/python310.lib

If I put the find_package(pybind11) first and don't link against Python3::Python, it seems to work correctly. This is contrary to the pybind11 documentation, however.

If I don't install the debug binaries then everything works correctly, and while I don't need the debug binaries, I would hope that their presence wouldn't lead to a linker error.

Something interesting I found is that pybind11 detects the python library by running PYTHON_EXECUTABLE and parsing the output. However, when you run find_package(Python) on a debug build, you get the release interpreter and the debug libraries, which I think is contributing to this issue.

Another interesting fact is that this bug only happens if you include one of pybind11's header files. If you just include Python.h, then it does not generate the linker error. (Which makes me feel like I'm going crazy)

Thank you for your time, I hope I've included enough information to be helpful!

Reproducible example code

* Download and install python from https://www.python.org/downloads/windows/
* Check `Download debug binaries`

CMakeLists.txt

CMAKE_MINIMUM_REQUIRED(VERSION 3.19)

PROJECT(Example)

FIND_PACKAGE(Python3 COMPONENTS Interpreter Development)
FIND_PACKAGE(pybind11 CONFIG)

ADD_EXECUTABLE(Example Main.cpp)

TARGET_LINK_LIBRARIES(Example PUBLIC Python3::Python pybind11::headers)

Main.cpp

#include <pybind11/embed.h>
namespace py = pybind11;

int main(int argc, char ** argv)
{
    py::scoped_interpreter guard{};
    py::print("Hello from pybind11");
    
    return 0;
}

[henryiii]

This looks like a possible bug in CMake's FindPython module in finding Debug libraries. No, wait a minute, you said

The find_package(Python) finds C:/Program Files/Python310/libs/python310_d.lib
The find_package(pybind11) finds C:/Program Files/Python310/libs/python310.lib

That's bizarre, it should not be looking for libraries again. And we should only be making links to the Python:: targets. PYTHON_IS_DEBUG will be wrong if it's pointing at the release executable but debug libraries. But that shouldn't invalidate the library it's linking to. There does not seem to be a way to get this information directly from FindPython, which would be much better.

Why does (only) Windows require linking to Python? :'(

[WhoBrokeTheBuild]

It's super strange, right? It's not exactly high priority, but I'm happy to provide any info / chat on gitter to help track it down.

[WhoBrokeTheBuild]

Oh, additionally, it only happens if I include the pybind11 headers.

For example, with the same CMake configuration:

// #include <pybind11/embed.h>
// namespace py = pybind11;

#include <Python.h>

int main(int argc, char ** argv)
{
    // py::scoped_interpreter guard{};
    // py::print("Hello from pybind11");

    Py_Initialize();
    Py_Finalize();
    
    return 0;
}

Will not trigger the linker error, which seems insane.

[henryiii]

I expect it would if you used more of the API? Py_Initialize/Py_Initialize might be identical?

Doesn't sound like FindPython supports debug mode windows very well, in the docs they mention you can ask for debug builds, but only on Unix. It's quite possible an issue needs to be opened with CMake. Though I would like to know what library is included in the Python::* targets to verify where the issue is happening.

[henryiii]

Is there a good way to test this locally? What's a good way to get a debug Windows build?

[WhoBrokeTheBuild]

That's a good point, I'm not sure.

You should be able to test it locally with the provided instructions. If you install the debug binaries, CMake will pick them up with find_package(Python). If you don't install them, then it finds the normal release libraries.

Here's the checkbox to click during Python's install:

I wrote some CMake to show what you were interested in:

GET_TARGET_PROPERTY(_lib Python3::Python IMPORTED_IMPLIB)
MESSAGE(STATUS "Python3::Python ${_lib}")

GET_TARGET_PROPERTY(_lib Python3::Python IMPORTED_IMPLIB_DEBUG)
MESSAGE(STATUS "Python3::Python (Debug) ${_lib}")

GET_TARGET_PROPERTY(_lib Python3::Python IMPORTED_IMPLIB_RELEASE)
MESSAGE(STATUS "Python3::Python (Release) ${_lib}")

Without the debug binaries installed:

[cmake] -- Python3::Python C:/Program Files/Python310/libs/python310.lib
[cmake] -- Python3::Python (Debug) _lib-NOTFOUND
[cmake] -- Python3::Python (Release) _lib-NOTFOUND

With the debug binaries installed:

[cmake] -- Python3::Python _lib-NOTFOUND
[cmake] -- Python3::Python (Debug) C:/Program Files/Python310/libs/python310_d.lib
[cmake] -- Python3::Python (Release) C:/Program Files/Python310/libs/python310.lib

I think that CMake's FindPython is doing what's expected, it properly finds both and marks them as debug/optimized appropriately.

Again, if you don't install the debug binaries it works fine, but this still seems like a stupid gotcha.

[henryiii]

What if you do Python3::Module instead? That should be what we link to. Are you building in debug configuration?

[henryiii]

And does Python3::Python (Release) C:/Program Files/Python310/libs/python310.lib exist?

[WhoBrokeTheBuild]

The result is the same with both Python3::Module and Python3::Embed

I am building with CMAKE_BUILD_TYPE=Debug, if I build in Release it does work however (I assume both Python and pybind11 choose the release version of python)

Yes, both python310.lib and python310_d.lib exist.

At worst, this could end up being a warning in the documentation, if there isn't a reasonable solution.

[henryiii]

I still don't see an obvious way it could be our fault:

$ git grep LIBRARIES -- tools/pybind11NewTools.cmake
tools/pybind11NewTools.cmake:    PROPERTY INTERFACE_LINK_LIBRARIES pybind11::python_headers)
tools/pybind11NewTools.cmake:    PROPERTY INTERFACE_LINK_LIBRARIES pybind11::python2_no_register)
tools/pybind11NewTools.cmake:    PROPERTY INTERFACE_LINK_LIBRARIES ${_Python}::Python)
tools/pybind11NewTools.cmake:    PROPERTY INTERFACE_LINK_LIBRARIES ${_Python}::Module)
tools/pybind11NewTools.cmake:    PROPERTY INTERFACE_LINK_LIBRARIES pybind11::python_link_helper)

We never use raw variables in the modern Python support. PY_IS_DEBUG is wrong, but that's okay, it doesn't do anything after Python 3.8. It should not be re-searching if Python3_FOUND is true.

The Include directories are from a raw variable, but that shouldn't change. We mostly wrap python3_add_module, I wonder if that is broken, actually? Checked the definition in CMake's source, looks fine, at least in the current master branch.

[bkloster-sma]

I also encountered this, but in a project that does not use CMake, so it is not a CMake issue.

I believe the culprit is this part of detail/common.h. The comment already gives a hint, but this is what I figure happens:

common.h undefined _DEBUG temporarily (it is redefined here) before including Python.h, which transitively includes pyconfig.h. And in pyconfig.h, you can find this (note the #pragma comment(lib, ...)):

/* For Windows the Python core is in a DLL by default.  Test
Py_NO_ENABLE_SHARED to find out.  Also support MS_NO_COREDLL for b/w compat */
#if !defined(MS_NO_COREDLL) && !defined(Py_NO_ENABLE_SHARED)
#       define Py_ENABLE_SHARED 1 /* standard symbol for shared library */
#       define MS_COREDLL       /* deprecated old symbol */
#endif /* !MS_NO_COREDLL && ... */

// [...]

/* For an MSVC DLL, we can nominate the .lib files used by extensions */
#ifdef MS_COREDLL
#       if !defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_BUILTIN)
                /* not building the core - must be an ext */
#               if defined(_MSC_VER)
                        /* So MSVC users need not specify the .lib
                        file in their Makefile (other compilers are
                        generally taken care of by distutils.) */
#                       if defined(_DEBUG)
#                               pragma comment(lib,"python310_d.lib")
#                       elif defined(Py_LIMITED_API)
#                               pragma comment(lib,"python3.lib")
#                       else
#                               pragma comment(lib,"python310.lib")
#                       endif /* _DEBUG */
#               endif /* _MSC_VER */
#       endif /* Py_BUILD_CORE */
#endif /* MS_COREDLL */

So without _DEBUG defined, pyconfig.h will attempt to link to python310.lib.

I haven't tried it out yet, but in theory, defining Py_NO_ENABLE_SHARED in the project should cause pyconfig.h to skip that entire section I quoted above. However, pybind11 would still #undef _DEBUG, even for builds genuinely linking against a Python debug builds, which might cause issues.

Would it be feasible to introduce a preprocessor option PYBIND11_DEBUG that, if defined, will cause _DEBUG not to be #undef'd?