Add expression evaluation in Presto sidecar #24126
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
To support constant folding and consistent semantics between the Presto coordinator (Java) and the Presto C++ worker, it is necessary to use consistent expression evaluation. To support this, a native expression evaluation endpoint,
v1/expressions
, has been added to the Presto sidecar process, and a plugin has been created which can utilize Velox expression evaluation behind a standardExpressionOptimizer
.Adds support to optimize and constant fold row expressions in the Presto native sidecar using class
RowExpressionOptimizer
. Adds theRowExpressionConverter
class to aidRowExpressionOptimizer
in converting the constant folded veloxexec::Expr
to a Prestoprotocol::RowExpression
.Motivation and Context
Consistency between C++ and Java semantics. Support for using C++ functions during constant folding of expressions in the planner.
Impact
Fix some lingering bugs with C++ clusters.
Test Plan
Tests have been added by extending the
TestRowExpressionInterpreter
class to also test native expression evaluation inTestNativeExpressionOptimizer.java
. However, this feature is still in Beta, and to support production workloads with complete certainty a fuzzer will be created to surface any remaining bugs with the integration at a later time.Release Notes