Skip to content

Commit 46182c8

Browse files
mertakmustafasrepoozankabak
authored
Replace repartition execs with sort preserving repartition execs (#6921)
* initialize replace_repartition_execs rule for SortPreservingRepartitionExec optimization typo fix * fix linter checks * use common idioms fix the inter children replacement issue simplify replacement logic use ordering_satisfy to compare orderings * fix ordering satisfy parameters according to provided output orderings * add Result wrappings on functions aggresively replace repartitions and do the check on final step * update RepartitionExec and HashJoinExec strings * update the documentation remove the unwrap method by using children() for sort exec * use .any for checking input order maintaining * Use existing order preserving RepartitionExec instead of dummy * Move rule to function * Simplifications * Remove sub-sule from enforce sorting * Run replace repartition for fixing pipeline * Comment improvements * Update comments * Better commenting * Address reviews * Change test so that it doesn't contain unnecessary executor --------- Co-authored-by: Mustafa Akur <[email protected]> Co-authored-by: Mehmet Ozan Kabak <[email protected]>
1 parent 44e3876 commit 46182c8

File tree

5 files changed

+805
-11
lines changed

5 files changed

+805
-11
lines changed

datafusion/core/src/physical_optimizer/mod.rs

+1
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ pub mod optimizer;
3131
pub mod pipeline_checker;
3232
pub mod pruning;
3333
pub mod repartition;
34+
pub mod replace_repartition_execs;
3435
pub mod sort_enforcement;
3536
mod sort_pushdown;
3637
mod utils;

0 commit comments

Comments
 (0)