Skip to content

Add benchmark for parquet reader with row_filter and project settings #7401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Apr 15, 2025
Merged

Add benchmark for parquet reader with row_filter and project settings #7401

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
822760c
Add benchmark for parquet reader with row_filter and project settings
zhuqi-lucas Apr 10, 2025
31a544f
fix clippy
zhuqi-lucas Apr 10, 2025
b16428d
change bench mark to use asyn read to trigger the page cache
zhuqi-lucas Apr 11, 2025
1aacd01
fix
zhuqi-lucas Apr 11, 2025
2d58006
Merge remote-tracking branch 'upstream/main' into benchmark_row_filter
zhuqi-lucas Apr 11, 2025
768826e
fix
zhuqi-lucas Apr 11, 2025
f624b91
Update comments, add background
alamb Apr 11, 2025
6c28e44
incremently addressing the comments
zhuqi-lucas Apr 11, 2025
69a2617
Fix bool random
zhuqi-lucas Apr 11, 2025
b044813
Merge commit '69a2617' into alamb/docs_for_bench
alamb Apr 11, 2025
6a37818
fixup
alamb Apr 11, 2025
2f6ccbb
Add fn switch and project enum
zhuqi-lucas Apr 11, 2025
994c747
Merge pull request #1 from alamb/alamb/docs_for_bench
zhuqi-lucas Apr 11, 2025
d0a656b
Fix clippy
zhuqi-lucas Apr 11, 2025
67480b9
Address comment
zhuqi-lucas Apr 12, 2025
16bc1bf
Add float(half set) and int(full set) change
zhuqi-lucas Apr 12, 2025
7638c41
Address comments
zhuqi-lucas Apr 13, 2025
9271cc9
Set compression
zhuqi-lucas Apr 14, 2025
8e00ac5
fix
zhuqi-lucas Apr 14, 2025
890519e
Update comments
alamb Apr 14, 2025
7eb0476
refactor filter column indexes
alamb Apr 14, 2025
22c7b39
Read from in memory buffer
alamb Apr 14, 2025
86878ab
Merge remote-tracking branch 'apache/main' into benchmark_row_filter
alamb Apr 14, 2025
5ae9b58
celanu
alamb Apr 14, 2025
1effe88
Test both sync and async readers
alamb Apr 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion parquet/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ ring = { version = "0.17", default-features = false, features = ["std"], optiona

[dev-dependencies]
base64 = { version = "0.22", default-features = false, features = ["std"] }
criterion = { version = "0.5", default-features = false }
criterion = { version = "0.5", default-features = false, features = ["async_futures"] }
snap = { version = "1.0", default-features = false }
tempfile = { version = "3.0", default-features = false }
brotli = { version = "7.0", default-features = false, features = ["std"] }
Expand Down Expand Up @@ -211,6 +211,10 @@ name = "arrow_statistics"
required-features = ["arrow"]
harness = false

[[bench]]
name = "arrow_reader_row_filter"
required-features = ["arrow", "async"]
harness = false

[[bench]]
name = "compression"
Expand Down
Loading
Loading