How to ship Cython pxd header files and other auxiliary files

[tobiasdiez]

Cython pxd files essentially function like C header files and you sometimes would like to ship them in the wheel. Similarly, sometimes you would like to include some other data files and expect them to be placed in the same directory as the python files. What is the best way to declare this in meson?

Option 1: py.install_sources: Listing them in install_sources along with the python files seems to work fine for production, in the sense that these header files are included in the final wheel or if one runs pip install. However, in editable install, the pxd files are not moved to the build folder and one thus, if one wants to run cython at runtime, needs to find out what was the original source directory to specify it as an include directory.

Option 2: fs.copyfile: This correctly copies the file for editable installs to the build directory, but doesn't include it in the wheel. There is an option install: true but then it asks for an install directory, which I don't know how to specify (a relative folder as with install_sources doesn't seem to work).

Thanks for your help!

[rgommers]

If you want to install data files (e.g., .pxd headers), then py.install_sources is indeed the easiest way.

For the second part: the reason you need to move .pxd files and __init__.py files to a build directory is that Cython looks at those by default to determine the correct name for the extension module it is building. There are two options to fix this:

1. fs.copyfile, as you pointed out. It's not an either/or with py.install_sources, you can use both (don't use install: true for fs.copyfile). We do this in SciPy.
2. cython takes a --module-name argument, which you can use to tell it the extension name (e.g., scipy.linalg._matfucs`); this should prevent Cython from trying to walk the directory tree.

[tobiasdiez]

Thanks! So the best solution at the moment would be to use copyfile in addition to install_sources?

[rgommers]

Yes, I think so. --module-name is possibly a little cleaner than using copyfile (since there won't be a need to copy __init__.py files as well), but I haven't used that recently so I'm not 100% sure. It's easy enough to try both on a single extension module I think, and see what you like better.