-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Refactor QueryPlanSerde to allow logic to be moved to individual classes per expression #1331
Conversation
@erenavsarogullari @NoeB @dharanad @jatin510 please let me know if you have any feedback on this approach |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @andygrove. it definitely improves the access and testability for methods which have been inner
import org.apache.comet.shims.CometExprShim | ||
|
||
trait CometExpression { | ||
def checkSupport(expr: Expression): Boolean | ||
trait CometExpressionSerde { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is presumably not just for arrays, so perhaps should be in a common file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, currently, there is only one implementation, which is why it is here at the moment, but this can move to a common class. Perhaps I should just move this into QueryPlanSerde source file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved the trait and added some scaladocs.
withInfo(expr, "unsupported arguments for GetArrayStructFields", child) | ||
None | ||
} | ||
case expr: ArrayRemove => CometArrayRemove.convert(expr, input, binding) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although there is a big diff, I suppose they are just code moving and this is the only change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, there are no functional changes. I just moved the inner methods to the top-level and that required adding extra parameters in some cases (such as input
and binding
).
*/ | ||
def exprToProto( | ||
expr: Expression, | ||
input: Seq[Attribute], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason to switching to singular input
from inputs
?
The doc above still says inputs
. Should we switch to plural inputs
since it is Seq
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In main, we currently have an inconsistency where exprToProto
uses input
and exprToProtoInternal
uses inputs
.
def exprToProto(
expr: Expression,
input: Seq[Attribute],
binding: Boolean = true): Option[Expr] = {
def exprToProtoInternal(expr: Expression, inputs: Seq[Attribute]): Option[Expr] = {
I changed both to input
for consistency, but I agree that inputs
is better. I have changed it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kazuyukitanimura fyi, I fixed the above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @andygrove
Which issue does this PR close?
Part of #1330
Rationale for this change
See #1330 for rationale.
What changes are included in this PR?
exprToProto
to top-level methods to make them accessible from other placesArrayRemove
serde to demonstrate the benefit of this refactorThe
ArrayRemove
code inQueryPlanSerde
now becomes:Later, we can move to a map of registered converters instead of a match statement for better extensibility. For example:
How are these changes tested?
There are no functional changes, so rely on existing tests.