You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
AggregateExpr currently provides both the ScalarValue based Accumulator and the new GroupsAccumulator interface. With an additional groups_accumulator_supported to indicate if GroupsAccumulator is supported.
This is unfortunate as it forces us to keep two accumulator implementations for each expression, which is not only wasteful but introduces the possibility of inconsistency between the two implementations
Describe the solution you'd like
Change AggregateExpr::create_accumulator to return GroupsAccumulator, and remove groups_accumulator_supported and create_groups_accumulator. Implementations that currently only have an Accumulator based implementation can return a GroupsAccumulatorAdapter, which is ultimately what GroupedHashAggregateStream will do anyway
I think updating AggregateExpr which is largely an internal implementation detail of DataFusion, to better communicate what is supported and what is not makes a lot of sense to me.
I think there are two accumulators now because Accumulator has an API to do retractable updates (incremental updates)
Accumulator is part of the public API so I don't think it can / should be removed
The only thing that might be tricky with this proposal is making sure the right implementation is provided for accumulators that can do retractable updates (like sum and count)
Is your feature request related to a problem or challenge?
AggregateExpr
currently provides both theScalarValue
basedAccumulator
and the newGroupsAccumulator
interface. With an additionalgroups_accumulator_supported
to indicate ifGroupsAccumulator
is supported.This is unfortunate as it forces us to keep two accumulator implementations for each expression, which is not only wasteful but introduces the possibility of inconsistency between the two implementations
Describe the solution you'd like
Change
AggregateExpr::create_accumulator
to returnGroupsAccumulator
, and removegroups_accumulator_supported
andcreate_groups_accumulator
. Implementations that currently only have anAccumulator
based implementation can return aGroupsAccumulatorAdapter
, which is ultimately whatGroupedHashAggregateStream
will do anywayDescribe alternatives you've considered
No response
Additional context
This blocks #6842
The text was updated successfully, but these errors were encountered: