cuDF-Polars tracks ordering metadata on streaming channels, but actor construction does not propagate downstream ordering requirements back to upstream operators. This means upstream sort, groupby, join, scan, and MapFunction("hint_sorted") nodes cannot make cost-aware decisions about preserving or materializing strict ordering needed by later operators.
Example:
q = (
pl.scan_parquet(path)
.set_sorted("ts")
.group_by("RIC", "ts")
.agg(pl.col("volume").sum())
.join_asof(other, on="ts", by="RIC")
)
In this kind of plan, the downstream as-of join benefits from strict ordering by (RIC, ts). The planner should be able to communicate that requirement upstream so:
MapFunction("hint_sorted") can decide whether extracting Parquet/order boundaries is worthwhile.
groupby / join actors can decide whether to preserve ordering, call adjust_ordering, or accept cheaper single-partition/tree behavior.
- sort actors can prioritize producing useful strict partitions when downstream consumers require them.
This is an ordering/planning optimization issue, not a new user-facing Polars API.
cuDF-Polars tracks ordering metadata on streaming channels, but actor construction does not propagate downstream ordering requirements back to upstream operators. This means upstream
sort,groupby,join, scan, andMapFunction("hint_sorted")nodes cannot make cost-aware decisions about preserving or materializing strict ordering needed by later operators.Example:
In this kind of plan, the downstream as-of join benefits from strict ordering by (RIC, ts). The planner should be able to communicate that requirement upstream so:
MapFunction("hint_sorted")can decide whether extracting Parquet/order boundaries is worthwhile.groupby/joinactors can decide whether to preserve ordering, calladjust_ordering, or accept cheaper single-partition/tree behavior.This is an ordering/planning optimization issue, not a new user-facing Polars API.