Skip to content

[FEA] Propagate downstream ordering requirements upward during lowering #23705

Description

@rjzamora

cuDF-Polars tracks ordering metadata on streaming channels, but actor construction does not propagate downstream ordering requirements back to upstream operators. This means upstream sort, groupby, join, scan, and MapFunction("hint_sorted") nodes cannot make cost-aware decisions about preserving or materializing strict ordering needed by later operators.

Example:

q = (
    pl.scan_parquet(path)
    .set_sorted("ts")
    .group_by("RIC", "ts")
    .agg(pl.col("volume").sum())
    .join_asof(other, on="ts", by="RIC")
)

In this kind of plan, the downstream as-of join benefits from strict ordering by (RIC, ts). The planner should be able to communicate that requirement upstream so:

  • MapFunction("hint_sorted") can decide whether extracting Parquet/order boundaries is worthwhile.
  • groupby / join actors can decide whether to preserve ordering, call adjust_ordering, or accept cheaper single-partition/tree behavior.
  • sort actors can prioritize producing useful strict partitions when downstream consumers require them.

This is an ordering/planning optimization issue, not a new user-facing Polars API.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions