Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: explore nanoarrow and sparrow #44

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

wgtmac
Copy link
Member

@wgtmac wgtmac commented Feb 8, 2025

Closes #33

@wgtmac
Copy link
Member Author

wgtmac commented Feb 9, 2025

I have managed to add sparrow as a vendored thirdparty dependency to libiceberg. However, there are still two issues that break CI:

  1. It cannot compile on Windows due to int128_t is not supported by MSVC: https://github.com/apache/iceberg-cpp/actions/runs/13217571682/job/36898649736?pr=44.
  2. It cannot compile on MacOS with AppleClang 15.0.0.15000309 since Sparrow README recommends Apple Clang 16 or higher: https://github.com/apache/iceberg-cpp/actions/runs/13217571682/job/36898649906?pr=44.

There are some other minor issues:

  1. It should use PREPEND not APPEND at this line: https://github.com/man-group/sparrow/blob/532bf486243fc396f7a9392820fa7a77071782ad/CMakeLists.txt#L50. Otherwise I have to add list(PREPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_BINARY_DIR}/_deps/sparrow-src/cmake) on my side when using FetchContent to vendor sparrow to avoid error include could not find requested file: sanitizers.
  2. It would be good to add a namespace Sparrow:: when installing the exported config file at https://github.com/man-group/sparrow/blob/532bf486243fc396f7a9392820fa7a77071782ad/CMakeLists.txt#L354.
  3. It should use target-oriented functions like target_compile_definition for https://github.com/man-group/sparrow/blob/532bf486243fc396f7a9392820fa7a77071782ad/CMakeLists.txt#L140 and similar places. Otherwise, projects depending on it are required to add same definitions in order to compile to avoid errors with regards to the date library.

These are not blocking issues at the moment. I need more time to get familiar with its API and best practice. Just want to report what I have found so far. @JohanMabille @Alex-PLACET @pitrou

@JohanMabille
Copy link

JohanMabille commented Feb 10, 2025

Hi there,

Thanks for the test and report! We got similar issues when trying to build and use sparrow in WASM and we are pushing fixes for that (more specifically, the target_compile_definitions that are missing). I'll take the opportunity to fix the other issues you faced, we will release a new version right after.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DISCUSS] Columnar data protocol: Arrow or implement a new one?
2 participants