Python fails to build on WASM: _testcapi/vectorcall_limited.c is built with Py_BUILD_CORE_BUILTIN

[vstinner]

My PR #109690 broke WASM buildbots.

Example with wasm32-emscripten node (dynamic linking) 3.x: https://buildbot.python.org/all/#/builders/1056/builds/3142

/opt/emsdk/upstream/emscripten/emcc  -DNDEBUG -g -O3 (...) -DPy_BUILD_CORE_BUILTIN -c ../../Modules/_testcapi/vectorcall_limited.c -o Modules/_testcapi/vectorcall_limited.o

In file included from ../../Modules/_testcapi/vectorcall_limited.c:2:
In file included from ../../Modules/_testcapi/parts.h:7:
In file included from ../../Include/Python.h:44:
../../Include/pyport.h:52:4: error: "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
#  error "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
   ^
1 error generated.

I'm not sure how -DPy_BUILD_CORE_BUILTIN landed in the command building Modules/_testcapi/vectorcall_limited.c.

I don't think that it's correct that vectorcall_limited.c which tests the limited C API on purpose it built with Py_BUILD_CORE.

On my Linux machine, Makefile contains:

Modules/_testcapi/vectorcall_limited.o: $(srcdir)/Modules/_testcapi/vectorcall_limited.c $(MODULE__TESTCAPI_DEPS) $(MODULE_DEPS_SHARED) $(PYTHON_HEADERS); $(CC) $(MODULE__TESTCAPI_CFLAGS) $(PY_STDMODULE_CFLAGS) $(CCSHARED) -c $(srcdir)/Modules/_testcapi/vectorcall_limited.c -o Modules/_testcapi/vectorcall_limited.o

So it gets two groups of compiler flags:

• MODULE__TESTCAPI_CFLAGS: not defined (empty)
• PY_STDMODULE_CFLAGS: -fno-strict-overflow -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -Og -Wall -O0 -std=c11 -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include

I don't see -DPy_BUILD_CORE_BUILTIN here.

I can reproduce the issue locally with this custom Modules/Setup.local:

*static*
_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/abstract.c _testcapi/unicode.c _testcapi/dict.c _testcapi/getargs.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyatomic.c _testcapi/pyos.c _testcapi/immortal.c _testcapi/heaptype_relative.c _testcapi/gc.c

make fails with:

In file included from ./Include/Python.h:44,
                 from ./Modules/_testcapi/parts.h:7,
                 from ./Modules/_testcapi/vectorcall_limited.c:2:
./Include/pyport.h:52:4: error: #error "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
   52 | #  error "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
      |    ^~~~~
make: *** [Makefile:2982: Modules/_testcapi/vectorcall_limited.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./Include/Python.h:44,
                 from ./Modules/_testcapi/parts.h:7,
                 from ./Modules/_testcapi/heaptype_relative.c:2:
./Include/pyport.h:52:4: error: #error "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
   52 | #  error "Py_LIMITED_API is not compatible with Py_BUILD_CORE"
      |    ^~~~~
make: *** [Makefile:3001: Modules/_testcapi/heaptype_relative.o] Error 1

Modules/makesetup uses $(PY_STDMODULE_CFLAGS) if $doconfig is no, but uses $(PY_BUILTIN_MODULE_CFLAGS) otherwise. In the second case, $(PY_BUILTIN_MODULE_CFLAGS) adds -DPy_BUILD_CORE_BUILTIN to compiler command used to build _testcapi C files.

Linked PRs

• gh-109723: Disable Py_BUILD_CORE in _testcapi #109727
• gh-109723: Fix build of _testclinic_limited on WASM #109842