Skip to content

DRAFT: Update arrow/parquet to 56.0.0 #16690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 5, 2025

Which issue does this PR close?

Rationale for this change

There are several non trivial changes in arrow 56 so I want to start testing soon

Also, I would like a stable base to test new parquet pushdown code from @XiangpengHao

What changes are included in this PR?

  1. Update to use pre-release version of arrow
  2. Update tests / APIs as necessary

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the common Related to common crate label Jul 5, 2025
@alamb alamb marked this pull request as draft July 5, 2025 11:57
@alamb alamb force-pushed the alamb/update_arrow_56.0.0 branch from 6698376 to 1747605 Compare July 5, 2025 12:20
@github-actions github-actions bot added logical-expr Logical plan and expressions proto Related to proto crate labels Jul 5, 2025
@alamb alamb force-pushed the alamb/update_arrow_56.0.0 branch from 1747605 to fc5bd79 Compare July 5, 2025 13:06
@github-actions github-actions bot added documentation Improvements or additions to documentation sql SQL Planner core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate datasource Changes to the datasource crate labels Jul 5, 2025
@alamb
Copy link
Contributor Author

alamb commented Jul 5, 2025

I see some failures in the row_group_pruning tests

$ cargo test --package datafusion --test parquet_config parquet::row_group_pruning
failures:
    parquet::row_group_pruning::prune_binary_eq_match
    parquet::row_group_pruning::prune_binary_lt
    parquet::row_group_pruning::prune_binary_neq
    parquet::row_group_pruning::prune_string_eq_match
    parquet::row_group_pruning::prune_string_lt
    parquet::row_group_pruning::prune_string_neq

I am tracking it down

Here is an example failure

thread 'parquet::row_group_pruning::prune_binary_eq_match' panicked at datafusion/core/tests/parquet/row_group_pruning.rs:146:9:
assertion `left == right` failed: Expected 2 rows, got 1: Input:
+---------------+----------------+------------------------------+-------------------+------------------------------+
| name          | service_string | service_binary               | service_fixedsize | service_large_binary         |
+---------------+----------------+------------------------------+-------------------+------------------------------+
| all frontends | frontend one   | 66726f6e74656e64206f6e65     | 666531            | 66726f6e74656e64206f6e65     |
| all frontends | frontend two   | 66726f6e74656e642074776f     | 666532            | 66726f6e74656e642074776f     |
| all frontends | frontend three | 66726f6e74656e64207468726565 | 666533            | 66726f6e74656e64207468726565 |
| all frontends | frontend seven | 66726f6e74656e6420736576656e | 666537            | 66726f6e74656e6420736576656e |
| all frontends | frontend five  | 66726f6e74656e642066697665   | 666535            | 66726f6e74656e642066697665   |
| mixed         | frontend six   | 66726f6e74656e6420736978     | 666536            | 66726f6e74656e6420736978     |
| mixed         | frontend four  | 66726f6e74656e6420666f7572   | 666534            | 66726f6e74656e6420666f7572   |
| mixed         | backend one    | 6261636b656e64206f6e65       | 626531            | 6261636b656e64206f6e65       |
| mixed         | backend two    | 6261636b656e642074776f       | 626532            | 6261636b656e642074776f       |
| mixed         | backend three  | 6261636b656e64207468726565   | 626533            | 6261636b656e64207468726565   |
| all backends  | backend four   | 6261636b656e6420666f7572     | 626534            | 6261636b656e6420666f7572     |
| all backends  | backend five   | 6261636b656e642066697665     | 626535            | 6261636b656e642066697665     |
| all backends  | backend six    | 6261636b656e6420736978       | 626536            | 6261636b656e6420736978       |
| all backends  | backend seven  | 6261636b656e6420736576656e   | 626537            | 6261636b656e6420736576656e   |
| all backends  | backend eight  | 6261636b656e64206569676874   | 626538            | 6261636b656e64206569676874   |
+---------------+----------------+------------------------------+-------------------+------------------------------+
Query:
SELECT name, service_binary FROM t WHERE service_binary = CAST('backend one' AS bytea)
Output:
+-------+------------------------+
| name  | service_binary         |
+-------+------------------------+
| mixed | 6261636b656e64206f6e65 |
| mixed | 6261636b656e642074776f |
+-------+------------------------+
Metrics:
time_elapsed_opening{partition=0}=51.348875ms, time_elapsed_scanning_until_data{partition=0}=179.834µs, time_elapsed_scanning_total{partition=0}=223.75µs, time_elapsed_processing{partition=0}=46.345583ms, file_open_errors{partition=0}=0, file_scan_errors{partition=0}=0, start_timestamp{partition=0}=2025-07-05 14:32:15.489561 UTC, end_timestamp{partition=0}=2025-07-05 14:32:15.541149 UTC, elapsed_compute{partition=0}=NOT RECORDED, output_rows{partition=0}=5, predicate_evaluation_errors{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, row_groups_pruned_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, row_groups_matched_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=2, row_groups_pruned_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, bytes_scanned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_pushdown_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, statistics_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=59.625µs, bloom_filter_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=46.298417ms, page_index_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=5, page_index_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=46.333µs, metadata_load_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=4.802583ms, files_ranges_pruned_statistics{partition=0}=0, predicate_evaluation_errors{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_pruned_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_pruned_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, bytes_scanned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=2098163, pushdown_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_pushdown_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, statistics_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, bloom_filter_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, page_index_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, metadata_load_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, files_ranges_pruned_statistics{partition=0}=0, num_predicate_creation_errors=0
  left: 2
 right: 1

@alamb
Copy link
Contributor Author

alamb commented Jul 7, 2025

ok, I now have a clean run!

@zhuqi-lucas
Copy link
Contributor

Thank you @alamb , I am curious about the benchmark result comparing the main branch, because we will include the apache/arrow-rs#7850 for this PR.

And some improvement part of the improvement we have ported to datafusion, but we will also benefit from the dependency changes from the arrow side, such as the sort phase(the merge compare we have ported)/compare, etc.

Could we trigger the benchmark for this PR, thanks!

@alamb
Copy link
Contributor Author

alamb commented Jul 8, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (76bc1b2) to ebb8e95 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Jul 8, 2025

Could we trigger the benchmark for this PR, thanks!

Done!

@alamb
Copy link
Contributor Author

alamb commented Jul 8, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  1968.78 ms │              1943.30 ms │    no change │
│ QQuery 1     │   671.98 ms │               752.37 ms │ 1.12x slower │
│ QQuery 2     │  1322.34 ms │              1460.81 ms │ 1.10x slower │
│ QQuery 3     │   679.14 ms │               685.65 ms │    no change │
│ QQuery 4     │  1379.83 ms │              1357.79 ms │    no change │
│ QQuery 5     │ 15240.77 ms │             15204.35 ms │    no change │
│ QQuery 6     │  2055.68 ms │              2095.06 ms │    no change │
│ QQuery 7     │  1832.29 ms │              1860.29 ms │    no change │
└──────────────┴─────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 25150.80ms │
│ Total Time (alamb_update_arrow_56.0)   │ 25359.62ms │
│ Average Time (HEAD)                    │  3143.85ms │
│ Average Time (alamb_update_arrow_56.0) │  3169.95ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │          6 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.50 ms │                 2.22 ms │ +1.13x faster │
│ QQuery 1     │    34.30 ms │                34.87 ms │     no change │
│ QQuery 2     │    80.44 ms │                82.83 ms │     no change │
│ QQuery 3     │    99.02 ms │                98.66 ms │     no change │
│ QQuery 4     │   604.00 ms │               591.53 ms │     no change │
│ QQuery 5     │   865.09 ms │               853.85 ms │     no change │
│ QQuery 6     │     2.22 ms │                 2.35 ms │  1.06x slower │
│ QQuery 7     │    38.66 ms │                39.53 ms │     no change │
│ QQuery 8     │   863.08 ms │               865.34 ms │     no change │
│ QQuery 9     │  1170.02 ms │              1198.50 ms │     no change │
│ QQuery 10    │   256.55 ms │               235.52 ms │ +1.09x faster │
│ QQuery 11    │   282.16 ms │               273.37 ms │     no change │
│ QQuery 12    │   856.42 ms │               879.42 ms │     no change │
│ QQuery 13    │  1251.28 ms │              1228.51 ms │     no change │
│ QQuery 14    │   810.79 ms │               801.54 ms │     no change │
│ QQuery 15    │   792.79 ms │               780.00 ms │     no change │
│ QQuery 16    │  1598.21 ms │              1612.46 ms │     no change │
│ QQuery 17    │  1600.07 ms │              1600.00 ms │     no change │
│ QQuery 18    │  2922.80 ms │              2859.57 ms │     no change │
│ QQuery 19    │    91.60 ms │                88.65 ms │     no change │
│ QQuery 20    │  1156.47 ms │              1178.00 ms │     no change │
│ QQuery 21    │  1291.31 ms │              1323.34 ms │     no change │
│ QQuery 22    │  2100.41 ms │              2199.67 ms │     no change │
│ QQuery 23    │  7358.55 ms │              7637.05 ms │     no change │
│ QQuery 24    │   438.16 ms │               415.34 ms │ +1.05x faster │
│ QQuery 25    │   299.70 ms │               287.79 ms │     no change │
│ QQuery 26    │   446.98 ms │               415.91 ms │ +1.07x faster │
│ QQuery 27    │  1534.24 ms │              1572.63 ms │     no change │
│ QQuery 28    │ 12754.23 ms │             11991.17 ms │ +1.06x faster │
│ QQuery 29    │   537.38 ms │               518.72 ms │     no change │
│ QQuery 30    │   777.26 ms │               778.63 ms │     no change │
│ QQuery 31    │   805.02 ms │               758.54 ms │ +1.06x faster │
│ QQuery 32    │  2408.21 ms │              2360.51 ms │     no change │
│ QQuery 33    │  3161.89 ms │              3169.50 ms │     no change │
│ QQuery 34    │  3181.61 ms │              3197.43 ms │     no change │
│ QQuery 35    │  1255.44 ms │              1256.22 ms │     no change │
│ QQuery 36    │   119.55 ms │               119.48 ms │     no change │
│ QQuery 37    │    49.89 ms │                50.74 ms │     no change │
│ QQuery 38    │   120.47 ms │               121.13 ms │     no change │
│ QQuery 39    │   199.39 ms │               200.24 ms │     no change │
│ QQuery 40    │    41.00 ms │                41.43 ms │     no change │
│ QQuery 41    │    38.10 ms │                38.31 ms │     no change │
│ QQuery 42    │    32.07 ms │                31.78 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 54329.35ms │
│ Total Time (alamb_update_arrow_56.0)   │ 53792.25ms │
│ Average Time (HEAD)                    │  1263.47ms │
│ Average Time (alamb_update_arrow_56.0) │  1250.98ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         36 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  98.26 ms │                95.79 ms │     no change │
│ QQuery 2     │  19.98 ms │                20.60 ms │     no change │
│ QQuery 3     │  32.60 ms │                32.88 ms │     no change │
│ QQuery 4     │  18.59 ms │                18.52 ms │     no change │
│ QQuery 5     │  48.91 ms │                49.15 ms │     no change │
│ QQuery 6     │  11.62 ms │                11.80 ms │     no change │
│ QQuery 7     │  90.21 ms │                87.97 ms │     no change │
│ QQuery 8     │  23.40 ms │                25.01 ms │  1.07x slower │
│ QQuery 9     │  53.14 ms │                54.06 ms │     no change │
│ QQuery 10    │  42.78 ms │                40.78 ms │     no change │
│ QQuery 11    │  11.41 ms │                11.28 ms │     no change │
│ QQuery 12    │  34.36 ms │                31.87 ms │ +1.08x faster │
│ QQuery 13    │  26.55 ms │                26.12 ms │     no change │
│ QQuery 14    │   9.73 ms │                 9.52 ms │     no change │
│ QQuery 15    │  18.69 ms │                18.55 ms │     no change │
│ QQuery 16    │  17.99 ms │                18.22 ms │     no change │
│ QQuery 17    │  96.57 ms │                95.77 ms │     no change │
│ QQuery 18    │ 186.92 ms │               198.21 ms │  1.06x slower │
│ QQuery 19    │  24.15 ms │                23.11 ms │     no change │
│ QQuery 20    │  31.03 ms │                31.32 ms │     no change │
│ QQuery 21    │ 142.83 ms │               144.34 ms │     no change │
│ QQuery 22    │  14.82 ms │                14.18 ms │     no change │
└──────────────┴───────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 1054.52ms │
│ Total Time (alamb_update_arrow_56.0)   │ 1059.05ms │
│ Average Time (HEAD)                    │   47.93ms │
│ Average Time (alamb_update_arrow_56.0) │   48.14ms │
│ Queries Faster                         │         1 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │        19 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

@zhuqi-lucas
Copy link
Contributor

zhuqi-lucas commented Jul 8, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  1968.78 ms │              1943.30 ms │    no change │
│ QQuery 1     │   671.98 ms │               752.37 ms │ 1.12x slower │
│ QQuery 2     │  1322.34 ms │              1460.81 ms │ 1.10x slower │
│ QQuery 3     │   679.14 ms │               685.65 ms │    no change │
│ QQuery 4     │  1379.83 ms │              1357.79 ms │    no change │
│ QQuery 5     │ 15240.77 ms │             15204.35 ms │    no change │
│ QQuery 6     │  2055.68 ms │              2095.06 ms │    no change │
│ QQuery 7     │  1832.29 ms │              1860.29 ms │    no change │
└──────────────┴─────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 25150.80ms │
│ Total Time (alamb_update_arrow_56.0)   │ 25359.62ms │
│ Average Time (HEAD)                    │  3143.85ms │
│ Average Time (alamb_update_arrow_56.0) │  3169.95ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │          6 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.50 ms │                 2.22 ms │ +1.13x faster │
│ QQuery 1     │    34.30 ms │                34.87 ms │     no change │
│ QQuery 2     │    80.44 ms │                82.83 ms │     no change │
│ QQuery 3     │    99.02 ms │                98.66 ms │     no change │
│ QQuery 4     │   604.00 ms │               591.53 ms │     no change │
│ QQuery 5     │   865.09 ms │               853.85 ms │     no change │
│ QQuery 6     │     2.22 ms │                 2.35 ms │  1.06x slower │
│ QQuery 7     │    38.66 ms │                39.53 ms │     no change │
│ QQuery 8     │   863.08 ms │               865.34 ms │     no change │
│ QQuery 9     │  1170.02 ms │              1198.50 ms │     no change │
│ QQuery 10    │   256.55 ms │               235.52 ms │ +1.09x faster │
│ QQuery 11    │   282.16 ms │               273.37 ms │     no change │
│ QQuery 12    │   856.42 ms │               879.42 ms │     no change │
│ QQuery 13    │  1251.28 ms │              1228.51 ms │     no change │
│ QQuery 14    │   810.79 ms │               801.54 ms │     no change │
│ QQuery 15    │   792.79 ms │               780.00 ms │     no change │
│ QQuery 16    │  1598.21 ms │              1612.46 ms │     no change │
│ QQuery 17    │  1600.07 ms │              1600.00 ms │     no change │
│ QQuery 18    │  2922.80 ms │              2859.57 ms │     no change │
│ QQuery 19    │    91.60 ms │                88.65 ms │     no change │
│ QQuery 20    │  1156.47 ms │              1178.00 ms │     no change │
│ QQuery 21    │  1291.31 ms │              1323.34 ms │     no change │
│ QQuery 22    │  2100.41 ms │              2199.67 ms │     no change │
│ QQuery 23    │  7358.55 ms │              7637.05 ms │     no change │
│ QQuery 24    │   438.16 ms │               415.34 ms │ +1.05x faster │
│ QQuery 25    │   299.70 ms │               287.79 ms │     no change │
│ QQuery 26    │   446.98 ms │               415.91 ms │ +1.07x faster │
│ QQuery 27    │  1534.24 ms │              1572.63 ms │     no change │
│ QQuery 28    │ 12754.23 ms │             11991.17 ms │ +1.06x faster │
│ QQuery 29    │   537.38 ms │               518.72 ms │     no change │
│ QQuery 30    │   777.26 ms │               778.63 ms │     no change │
│ QQuery 31    │   805.02 ms │               758.54 ms │ +1.06x faster │
│ QQuery 32    │  2408.21 ms │              2360.51 ms │     no change │
│ QQuery 33    │  3161.89 ms │              3169.50 ms │     no change │
│ QQuery 34    │  3181.61 ms │              3197.43 ms │     no change │
│ QQuery 35    │  1255.44 ms │              1256.22 ms │     no change │
│ QQuery 36    │   119.55 ms │               119.48 ms │     no change │
│ QQuery 37    │    49.89 ms │                50.74 ms │     no change │
│ QQuery 38    │   120.47 ms │               121.13 ms │     no change │
│ QQuery 39    │   199.39 ms │               200.24 ms │     no change │
│ QQuery 40    │    41.00 ms │                41.43 ms │     no change │
│ QQuery 41    │    38.10 ms │                38.31 ms │     no change │
│ QQuery 42    │    32.07 ms │                31.78 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 54329.35ms │
│ Total Time (alamb_update_arrow_56.0)   │ 53792.25ms │
│ Average Time (HEAD)                    │  1263.47ms │
│ Average Time (alamb_update_arrow_56.0) │  1250.98ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         36 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  98.26 ms │                95.79 ms │     no change │
│ QQuery 2     │  19.98 ms │                20.60 ms │     no change │
│ QQuery 3     │  32.60 ms │                32.88 ms │     no change │
│ QQuery 4     │  18.59 ms │                18.52 ms │     no change │
│ QQuery 5     │  48.91 ms │                49.15 ms │     no change │
│ QQuery 6     │  11.62 ms │                11.80 ms │     no change │
│ QQuery 7     │  90.21 ms │                87.97 ms │     no change │
│ QQuery 8     │  23.40 ms │                25.01 ms │  1.07x slower │
│ QQuery 9     │  53.14 ms │                54.06 ms │     no change │
│ QQuery 10    │  42.78 ms │                40.78 ms │     no change │
│ QQuery 11    │  11.41 ms │                11.28 ms │     no change │
│ QQuery 12    │  34.36 ms │                31.87 ms │ +1.08x faster │
│ QQuery 13    │  26.55 ms │                26.12 ms │     no change │
│ QQuery 14    │   9.73 ms │                 9.52 ms │     no change │
│ QQuery 15    │  18.69 ms │                18.55 ms │     no change │
│ QQuery 16    │  17.99 ms │                18.22 ms │     no change │
│ QQuery 17    │  96.57 ms │                95.77 ms │     no change │
│ QQuery 18    │ 186.92 ms │               198.21 ms │  1.06x slower │
│ QQuery 19    │  24.15 ms │                23.11 ms │     no change │
│ QQuery 20    │  31.03 ms │                31.32 ms │     no change │
│ QQuery 21    │ 142.83 ms │               144.34 ms │     no change │
│ QQuery 22    │  14.82 ms │                14.18 ms │     no change │
└──────────────┴───────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 1054.52ms │
│ Total Time (alamb_update_arrow_56.0)   │ 1059.05ms │
│ Average Time (HEAD)                    │   47.93ms │
│ Average Time (alamb_update_arrow_56.0) │   48.14ms │
│ Queries Faster                         │         1 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │        19 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

Thank you @alamb, it seems we have some improvement for clickbench. Not too much because we gain for sort string view mostly which is not in clickbench but in sort_tpch.

@alamb
Copy link
Contributor Author

alamb commented Jul 8, 2025

Thank you @alamb, it seems we have some improvement for clickbench. Not too much because we gain for sort string view mostly which is not in clickbench but in sort_tpch.

I will start those as well

@alamb

This comment was marked as outdated.

@alamb

This comment was marked as outdated.

@zhuqi-lucas
Copy link
Contributor

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ Q1           │  330.29 ms │               350.13 ms │ 1.06x slower │
│ Q2           │  308.58 ms │               312.59 ms │    no change │
│ Q3           │ 1051.46 ms │              1076.53 ms │    no change │
│ Q4           │  444.92 ms │               462.29 ms │    no change │
│ Q5           │  410.39 ms │               412.36 ms │    no change │
│ Q6           │  449.16 ms │               452.76 ms │    no change │
│ Q7           │  783.43 ms │               833.85 ms │ 1.06x slower │
│ Q8           │  696.74 ms │               712.94 ms │    no change │
│ Q9           │  732.56 ms │               732.64 ms │    no change │
│ Q10          │ 1077.29 ms │              1090.28 ms │    no change │
│ Q11          │  547.35 ms │               563.97 ms │    no change │
└──────────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6832.16ms │
│ Total Time (alamb_update_arrow_56.0)   │ 7000.33ms │
│ Average Time (HEAD)                    │  621.11ms │
│ Average Time (alamb_update_arrow_56.0) │  636.39ms │
│ Queries Faster                         │         0 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │         9 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

Thank you @alamb , it seems no improvement for sort dependencies(So it means the merge will occupied most time), so we gain most for the ported PRs to merge phase from:

#16509
#16630

@alamb

This comment was marked as outdated.

@alamb

This comment was marked as outdated.

@zhuqi-lucas
Copy link
Contributor

zhuqi-lucas commented Jul 9, 2025

Thank you @alamb @Dandandan , we may also try sort_tpch10 benchmark , but it may also not too much improvement, the ported PR already has 1.4x faster for sort_tpch Q11(inlined string view sort).

@Dandandan
Copy link
Contributor

I wonder if we should focus on more parallelisation in SortPreservingMerge (besides improving the hot paths like gc interleave, comparisons, etc.)

I think there might be two areas:

  • Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.
  • Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

@zhuqi-lucas
Copy link
Contributor

I wonder if we should focus on more parallelisation in SortPreservingMerge (besides improving the hot paths like gc interleave, comparisons, etc.)

I think there might be two areas:

  • Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.
  • Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

Thank you @Dandandan , interesting idea, i agree if we can improve this point, it will help not only stringview both also other type, so it will benefit more cases, i will investigate!

@zhuqi-lucas
Copy link
Contributor

@alamb @Dandandan

I submit a PR based this PR, try to see if fast_gc can also help for sort_tpch or sort_tpch10 benchmark:

#16739

May be we can run sort_tpch for above PR to see the result, thanks!

@alamb
Copy link
Contributor Author

alamb commented Jul 11, 2025

  • Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.

Here is a related idea;

  • Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

This is an interesting idea and I think it may make a significant difference -- the core merge is a single threaded operation, so the less time that thread has to wait (for example, to fetch the next cursor of rows) will contribute directly to the overall speed of the query

@alamb alamb force-pushed the alamb/update_arrow_56.0.0 branch from 76bc1b2 to aa4b8cb Compare July 15, 2025 17:57
@alamb

This comment was marked as outdated.

@alamb

This comment was marked as outdated.

@alamb

This comment was marked as outdated.

@alamb alamb force-pushed the alamb/update_arrow_56.0.0 branch from 1f4876a to 0a0c7d7 Compare July 29, 2025 16:50
@alamb
Copy link
Contributor Author

alamb commented Jul 29, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (0a0c7d7) to cca9d4c diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Jul 29, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2029.57 ms │              2159.42 ms │  1.06x slower │
│ QQuery 1     │   748.35 ms │               744.68 ms │     no change │
│ QQuery 2     │  1455.54 ms │              1376.28 ms │ +1.06x faster │
│ QQuery 3     │   679.84 ms │               692.55 ms │     no change │
│ QQuery 4     │  1389.54 ms │              1373.55 ms │     no change │
│ QQuery 5     │ 14690.39 ms │             14869.06 ms │     no change │
│ QQuery 6     │  2045.96 ms │              2303.99 ms │  1.13x slower │
│ QQuery 7     │  1990.40 ms │              1969.12 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 25029.60ms │
│ Total Time (alamb_update_arrow_56.0)   │ 25488.66ms │
│ Average Time (HEAD)                    │  3128.70ms │
│ Average Time (alamb_update_arrow_56.0) │  3186.08ms │
│ Queries Faster                         │          1 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │          5 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.25 ms │                 2.30 ms │     no change │
│ QQuery 1     │    34.18 ms │                33.74 ms │     no change │
│ QQuery 2     │    82.92 ms │                80.08 ms │     no change │
│ QQuery 3     │   103.92 ms │                98.07 ms │ +1.06x faster │
│ QQuery 4     │   597.08 ms │               607.67 ms │     no change │
│ QQuery 5     │   830.16 ms │               841.77 ms │     no change │
│ QQuery 6     │     2.23 ms │                 2.34 ms │  1.05x slower │
│ QQuery 7     │    38.36 ms │                39.62 ms │     no change │
│ QQuery 8     │   847.13 ms │               866.56 ms │     no change │
│ QQuery 9     │  1150.17 ms │              1125.46 ms │     no change │
│ QQuery 10    │   259.68 ms │               231.19 ms │ +1.12x faster │
│ QQuery 11    │   286.48 ms │               258.83 ms │ +1.11x faster │
│ QQuery 12    │   857.98 ms │               838.43 ms │     no change │
│ QQuery 13    │  1113.39 ms │              1191.98 ms │  1.07x slower │
│ QQuery 14    │   803.84 ms │               789.50 ms │     no change │
│ QQuery 15    │   760.28 ms │               781.69 ms │     no change │
│ QQuery 16    │  1572.65 ms │              1595.98 ms │     no change │
│ QQuery 17    │  1574.09 ms │              1598.80 ms │     no change │
│ QQuery 18    │  2795.51 ms │              2852.72 ms │     no change │
│ QQuery 19    │    86.68 ms │                85.04 ms │     no change │
│ QQuery 20    │  1155.67 ms │              1124.69 ms │     no change │
│ QQuery 21    │  1282.39 ms │              1265.08 ms │     no change │
│ QQuery 22    │  2129.17 ms │              2099.33 ms │     no change │
│ QQuery 23    │  7347.85 ms │              7317.42 ms │     no change │
│ QQuery 24    │   440.32 ms │               399.16 ms │ +1.10x faster │
│ QQuery 25    │   298.74 ms │               266.57 ms │ +1.12x faster │
│ QQuery 26    │   436.21 ms │               399.06 ms │ +1.09x faster │
│ QQuery 27    │  1543.68 ms │              1508.85 ms │     no change │
│ QQuery 28    │ 12002.06 ms │             11744.39 ms │     no change │
│ QQuery 29    │   524.33 ms │               530.73 ms │     no change │
│ QQuery 30    │   761.94 ms │               752.14 ms │     no change │
│ QQuery 31    │   783.83 ms │               774.72 ms │     no change │
│ QQuery 32    │  2421.79 ms │              2498.52 ms │     no change │
│ QQuery 33    │  3262.10 ms │              3166.04 ms │     no change │
│ QQuery 34    │  3340.61 ms │              3209.38 ms │     no change │
│ QQuery 35    │  1285.68 ms │              1307.29 ms │     no change │
│ QQuery 36    │   122.03 ms │               119.48 ms │     no change │
│ QQuery 37    │    52.64 ms │                52.46 ms │     no change │
│ QQuery 38    │   118.16 ms │               117.67 ms │     no change │
│ QQuery 39    │   195.25 ms │               187.97 ms │     no change │
│ QQuery 40    │    41.62 ms │                41.75 ms │     no change │
│ QQuery 41    │    39.26 ms │                38.55 ms │     no change │
│ QQuery 42    │    33.69 ms │                33.09 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 53417.99ms │
│ Total Time (alamb_update_arrow_56.0)   │ 52876.11ms │
│ Average Time (HEAD)                    │  1242.28ms │
│ Average Time (alamb_update_arrow_56.0) │  1229.68ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │         35 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  97.13 ms │                97.30 ms │     no change │
│ QQuery 2     │  20.67 ms │                21.05 ms │     no change │
│ QQuery 3     │  32.23 ms │                31.77 ms │     no change │
│ QQuery 4     │  18.33 ms │                18.49 ms │     no change │
│ QQuery 5     │  49.61 ms │                48.73 ms │     no change │
│ QQuery 6     │  11.90 ms │                11.78 ms │     no change │
│ QQuery 7     │  85.99 ms │                83.46 ms │     no change │
│ QQuery 8     │  24.83 ms │                23.36 ms │ +1.06x faster │
│ QQuery 9     │  53.65 ms │                52.90 ms │     no change │
│ QQuery 10    │  42.02 ms │                40.18 ms │     no change │
│ QQuery 11    │  11.14 ms │                11.03 ms │     no change │
│ QQuery 12    │  35.95 ms │                31.26 ms │ +1.15x faster │
│ QQuery 13    │  26.05 ms │                25.83 ms │     no change │
│ QQuery 14    │   9.69 ms │                 9.81 ms │     no change │
│ QQuery 15    │  19.02 ms │                19.30 ms │     no change │
│ QQuery 16    │  17.94 ms │                17.78 ms │     no change │
│ QQuery 17    │  95.41 ms │                92.78 ms │     no change │
│ QQuery 18    │ 190.80 ms │               186.22 ms │     no change │
│ QQuery 19    │  24.70 ms │                23.66 ms │     no change │
│ QQuery 20    │  31.29 ms │                31.81 ms │     no change │
│ QQuery 21    │ 142.09 ms │               139.93 ms │     no change │
│ QQuery 22    │  14.69 ms │                14.35 ms │     no change │
└──────────────┴───────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 1055.14ms │
│ Total Time (alamb_update_arrow_56.0)   │ 1032.80ms │
│ Average Time (HEAD)                    │   47.96ms │
│ Average Time (alamb_update_arrow_56.0) │   46.95ms │
│ Queries Faster                         │         2 │
│ Queries Slower                         │         0 │
│ Queries with No Change                 │        20 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

@alamb
Copy link
Contributor Author

alamb commented Jul 29, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (0a0c7d7) to cca9d4c diff using: clickbench_pushdown
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Jul 29, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.29 ms │                 2.72 ms │  1.19x slower │
│ QQuery 1     │    36.92 ms │                36.58 ms │     no change │
│ QQuery 2     │    84.01 ms │                80.19 ms │     no change │
│ QQuery 3     │   104.07 ms │                97.29 ms │ +1.07x faster │
│ QQuery 4     │   600.76 ms │               608.71 ms │     no change │
│ QQuery 5     │   858.30 ms │               834.00 ms │     no change │
│ QQuery 6     │     2.34 ms │                 2.25 ms │     no change │
│ QQuery 7     │    56.18 ms │                56.85 ms │     no change │
│ QQuery 8     │   851.28 ms │               860.37 ms │     no change │
│ QQuery 9     │  1186.48 ms │              1176.20 ms │     no change │
│ QQuery 10    │   282.63 ms │               256.76 ms │ +1.10x faster │
│ QQuery 11    │   325.74 ms │               301.51 ms │ +1.08x faster │
│ QQuery 12    │  1074.47 ms │              1041.40 ms │     no change │
│ QQuery 13    │  1541.46 ms │              1509.21 ms │     no change │
│ QQuery 14    │  1120.02 ms │              1088.89 ms │     no change │
│ QQuery 15    │   769.78 ms │               785.80 ms │     no change │
│ QQuery 16    │  1625.63 ms │              1599.02 ms │     no change │
│ QQuery 17    │  1600.57 ms │              1586.90 ms │     no change │
│ QQuery 18    │  2935.97 ms │              2877.02 ms │     no change │
│ QQuery 19    │    87.35 ms │                85.46 ms │     no change │
│ QQuery 20    │  1159.03 ms │              1106.38 ms │     no change │
│ QQuery 21    │  1338.02 ms │              1295.42 ms │     no change │
│ QQuery 22    │  2525.18 ms │              2436.06 ms │     no change │
│ QQuery 23    │   880.18 ms │               940.44 ms │  1.07x slower │
│ QQuery 24    │   232.42 ms │               215.78 ms │ +1.08x faster │
│ QQuery 25    │   562.49 ms │               523.07 ms │ +1.08x faster │
│ QQuery 26    │   406.97 ms │               369.07 ms │ +1.10x faster │
│ QQuery 27    │  2252.79 ms │              2216.17 ms │     no change │
│ QQuery 28    │ 12613.00 ms │             12576.07 ms │     no change │
│ QQuery 29    │   525.64 ms │               527.57 ms │     no change │
│ QQuery 30    │  1236.81 ms │              1183.96 ms │     no change │
│ QQuery 31    │  1207.72 ms │              1209.10 ms │     no change │
│ QQuery 32    │  2542.60 ms │              2422.82 ms │     no change │
│ QQuery 33    │  3244.30 ms │              3209.52 ms │     no change │
│ QQuery 34    │  3288.77 ms │              3184.94 ms │     no change │
│ QQuery 35    │  1301.21 ms │              1284.38 ms │     no change │
│ QQuery 36    │    29.71 ms │                29.90 ms │     no change │
│ QQuery 37    │    27.84 ms │                27.74 ms │     no change │
│ QQuery 38    │    27.41 ms │                28.49 ms │     no change │
│ QQuery 39    │    27.78 ms │                28.07 ms │     no change │
│ QQuery 40    │    27.70 ms │                28.31 ms │     no change │
│ QQuery 41    │    28.15 ms │                27.73 ms │     no change │
│ QQuery 42    │    27.52 ms │                28.37 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 50659.45ms │
│ Total Time (alamb_update_arrow_56.0)   │ 49786.48ms │
│ Average Time (HEAD)                    │  1178.13ms │
│ Average Time (alamb_update_arrow_56.0) │  1157.83ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │         35 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

@zhuqi-lucas
Copy link
Contributor

Thank you @alamb , the performance result looks like we gain some performance from apache/arrow-rs#7802 now. We may also trigger the sort-tpch or sort-tpch10 to see if we gain some performance now even we have ported part of them in datafusion, thanks!

@alamb
Copy link
Contributor Author

alamb commented Jul 30, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (0a0c7d7) to cca9d4c diff using: sort_tpch
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Jul 30, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Q1           │  349.61 ms │               323.45 ms │ +1.08x faster │
│ Q2           │  308.94 ms │               317.36 ms │     no change │
│ Q3           │ 1112.50 ms │              1014.45 ms │ +1.10x faster │
│ Q4           │  437.16 ms │               431.15 ms │     no change │
│ Q5           │  418.20 ms │               416.31 ms │     no change │
│ Q6           │  459.93 ms │               453.85 ms │     no change │
│ Q7           │  817.52 ms │               817.33 ms │     no change │
│ Q8           │  705.87 ms │               698.17 ms │     no change │
│ Q9           │  734.30 ms │               724.44 ms │     no change │
│ Q10          │ 1084.13 ms │              1057.24 ms │     no change │
│ Q11          │  569.35 ms │               552.48 ms │     no change │
└──────────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6997.49ms │
│ Total Time (alamb_update_arrow_56.0)   │ 6806.22ms │
│ Average Time (HEAD)                    │  636.14ms │
│ Average Time (alamb_update_arrow_56.0) │  618.75ms │
│ Queries Faster                         │         2 │
│ Queries Slower                         │         0 │
│ Queries with No Change                 │         9 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

@alamb
Copy link
Contributor Author

alamb commented Jul 30, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (0a0c7d7) to cca9d4c diff using: sort_tpch10
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Jul 30, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch10.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃ HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ Q1           │ FAIL │                    FAIL │ incomparable │
│ Q2           │ FAIL │                    FAIL │ incomparable │
│ Q3           │ FAIL │                    FAIL │ incomparable │
│ Q4           │ FAIL │                    FAIL │ incomparable │
│ Q5           │ FAIL │                    FAIL │ incomparable │
│ Q6           │ FAIL │                    FAIL │ incomparable │
│ Q7           │ FAIL │                    FAIL │ incomparable │
│ Q8           │ FAIL │                    FAIL │ incomparable │
│ Q9           │ FAIL │                    FAIL │ incomparable │
│ Q10          │ FAIL │                    FAIL │ incomparable │
│ Q11          │ FAIL │                    FAIL │ incomparable │
└──────────────┴──────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ Benchmark Summary                      ┃        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ Total Time (HEAD)                      │ 0.00ms │
│ Total Time (alamb_update_arrow_56.0)   │ 0.00ms │
│ Average Time (HEAD)                    │ 0.00ms │
│ Average Time (alamb_update_arrow_56.0) │ 0.00ms │
│ Queries Faster                         │      0 │
│ Queries Slower                         │      0 │
│ Queries with No Change                 │      0 │
│ Queries with Failure                   │     11 │
└────────────────────────────────────────┴────────┘

@zhuqi-lucas
Copy link
Contributor

zhuqi-lucas commented Jul 31, 2025

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Q1           │  349.61 ms │               323.45 ms │ +1.08x faster │
│ Q2           │  308.94 ms │               317.36 ms │     no change │
│ Q3           │ 1112.50 ms │              1014.45 ms │ +1.10x faster │
│ Q4           │  437.16 ms │               431.15 ms │     no change │
│ Q5           │  418.20 ms │               416.31 ms │     no change │
│ Q6           │  459.93 ms │               453.85 ms │     no change │
│ Q7           │  817.52 ms │               817.33 ms │     no change │
│ Q8           │  705.87 ms │               698.17 ms │     no change │
│ Q9           │  734.30 ms │               724.44 ms │     no change │
│ Q10          │ 1084.13 ms │              1057.24 ms │     no change │
│ Q11          │  569.35 ms │               552.48 ms │     no change │
└──────────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6997.49ms │
│ Total Time (alamb_update_arrow_56.0)   │ 6806.22ms │
│ Average Time (HEAD)                    │  636.14ms │
│ Average Time (alamb_update_arrow_56.0) │  618.75ms │
│ Queries Faster                         │         2 │
│ Queries Slower                         │         0 │
│ Queries with No Change                 │         9 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

Thank you @alamb the result is very good, we have further improvement for sort also besides the ported code in datafusion, even Q1 is benefit which is not StringView sort from the new partition null validation, and Q3 is benefit from the new GC for large string size. And the sort-tpch10 seems broken. 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
common Related to common crate core Core DataFusion crate datasource Changes to the datasource crate documentation Improvements or additions to documentation logical-expr Logical plan and expressions proto Related to proto crate sql SQL Planner sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants