Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add expression evaluation in Presto sidecar #24126

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Conversation

tdcmeehan
Copy link
Contributor

@tdcmeehan tdcmeehan commented Nov 22, 2024

Description

To support constant folding and consistent semantics between the Presto coordinator (Java) and the Presto C++ worker, it is necessary to use consistent expression evaluation. To support this, a native expression evaluation endpoint, v1/expressions, has been added to the Presto sidecar process, and a plugin has been created which can utilize Velox expression evaluation behind a standard ExpressionOptimizer.

Adds support to optimize and constant fold row expressions in the Presto native sidecar using class RowExpressionOptimizer. Adds the RowExpressionConverter class to aid RowExpressionOptimizer in converting the constant folded velox exec::Expr to a Presto protocol::RowExpression.

Motivation and Context

Consistency between C++ and Java semantics. Support for using C++ functions during constant folding of expressions in the planner.

Impact

Fix some lingering bugs with C++ clusters.

Test Plan

Tests have been added by extending the TestRowExpressionInterpreter class to also test native expression evaluation in TestNativeExpressionOptimizer.java. However, this feature is still in Beta, and to support production workloads with complete certainty a fuzzer will be created to surface any remaining bugs with the integration at a later time.

Release Notes

== RELEASE NOTES ==

General Changes
* ... :pr:`12345`
* ... :pr:`12345`

Hive Connector Changes
* ... :pr:`12345`
* ... :pr:`12345`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
from:IBM PR from IBM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants