Skip to content

Conversation

@JAicewizard
Copy link
Contributor

Fixes #159

This propperly implements cross-compiling for the c++/c extensions, using toolchain files for each target. This allows CMake to detect when it is cross-compiling, and act accordingly. For example, CMake will set the CMAKE_CROSS_COMPILING variable, which can be used by extensions as intended. The faiss extension started to support CUDA recently, and this needs CMAKE_CROSS_COMPILING to be set for it to detect the correct libraries. Currently this is done manually in our Makefile, but this is suboptimal.

The biggest benefit is the fact that all the variables are now in one place, they are no longer split between the yaml file, and the makefile. For example the rust and OSX variables were in the Makefile, but all flags relates to c and c++ were in the yaml. Having a unified location also makes adding toolchains easier. Adding the variables for CUDA would either turn the makefile into a long list of checks and a mess of variables. Being able to do this in CMake is both easier and cleaner.

@JAicewizard
Copy link
Contributor Author

@carlopi if there is anything against this LMK, then I will start making a PR for adding a cuda toolchain without the CMake files. I do however believe having files like this is the best for consistent duckdb (extension) compilation, and a clean way to make CMake aware of the necessary toolchain options. (Potentially, we could even fix more of the options, like always specifying exact gcc/rustc toolchains exactly to make building even more consistent.)

I noticed we also rename the git linker, I think we could also specify the linker in CMake, so potentially that hack can go too.

@carlopi
Copy link
Collaborator

carlopi commented Apr 15, 2025

Thanks @JAicewizard! This looks to make a lot of sense, but there is quite some details, I will try to do a proper review.

@carlopi carlopi self-requested a review April 15, 2025 08:34
@JAicewizard JAicewizard force-pushed the toolchainfiles branch 3 times, most recently from dcaeeb5 to 455c4dc Compare June 27, 2025 21:40
@JAicewizard
Copy link
Contributor Author

CI Seems to be running propperly again. I just rebased this, could you take another look at this @carlopi?

The switch to almalinux on arm64 triggered some issues, since I assumed arm64 would be cross-compiled. However, this should now be fixed, and arm64 works without (and with) cross-compiling.

Sorry for the commit spam, I had to try some things but I found the issue and cleaned the history

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow CMAKE to detect cross-compiling

2 participants