Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partitioning: Use pyiceberg_core transform for all pyarrow transforms #1738

Open
kevinjqliu opened this issue Feb 27, 2025 · 1 comment
Open

Comments

@kevinjqliu
Copy link
Contributor

Feature Request / Improvement

We expose pyiceberg_core's transform functions (powered by iceberg-rust) for partition transform
#1074 and #1345 added this functionality for bucket and truncate transform.

Let's expand to that include all other transformation types.

The assumption is that pyiceberg_core will be faster, but we should benchmark it

cc @sungwy

@sungwy
Copy link
Collaborator

sungwy commented Feb 27, 2025

I'd love to see some benchmarks 📊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants