-
Notifications
You must be signed in to change notification settings - Fork 480
Avoid deconstructing temporal filters in memoize_expr
#33500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
030fda2
to
254bc48
Compare
src/expr/src/linear.rs
Outdated
soft_assert_or_log!( | ||
!is_temporal_filter, | ||
"MfpPlan::create_from and is_temporal_filter should agree" | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm also open to refactoring these into one method if deemed worthwhile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Eh, seems fine as is. Maybe worth adding a comment that temporal filters are necessarily binary operations?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Eh, seems fine as is.
Ok, I'm relieved :)
Maybe worth adding a comment that temporal filters are necessarily binary operations?
Below the let is_temporal_filter =
line, there is the following old comment:
// Supported temporal predicates are exclusively binary operators.
Or did you mean in the doc comment of the fn is_temporal_filter
? That kinda has it already, albeit somewhat implicitly, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I ended up factoring this out, due to Frank's comment.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nits---looks good to me!
src/expr/src/linear.rs
Outdated
soft_assert_or_log!( | ||
!is_temporal_filter, | ||
"MfpPlan::create_from and is_temporal_filter should agree" | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Eh, seems fine as is. Maybe worth adding a comment that temporal filters are necessarily binary operations?
254bc48
to
06ee435
Compare
To discuss: why not use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless there is a rush, I think there's potentially a clearer way to do this, where we centralize the "am I a temporal filter" logic, deduplicating that and removing the need for tests that the two bits of logic agree.
src/expr/src/scalar.rs
Outdated
/// - no mz_now() anywhere inside the other side of the comparison. | ||
/// | ||
/// If yes, then it returns the other side of the comparison. | ||
pub fn is_temporal_filter(&mut self) -> Option<&mut MirScalarExpr> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit, but naming-wise I would expect is_foo(&mut self) -> ...
to return a bool and not be &mut
. Perhaps instead as_mut_temporal_filter_expr
? That's a bit horrid, but probably clearer when used and less surprising.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading the other changes, I think what we actually want here is
/// If `self` expresses a temporal filter, normalize it to start with `mz_now()` and return references.
///
/// A temporal filter is an expression of the form `mz_now() <BINOP> <EXPR>`,
/// for a restricted set of `BINOP` and `EXPR` that do not themselves contain `mz_now()`.
/// Expressions may conform to this once their expressions are swapped.
///
/// If the expression is not a temporal filter it will be unchanged.
pub fn as_mut_temporal_filter(&mut self) -> Option<(&BinaryFunc, &mut MirScalarExpr)>
which I think centralizes the logic in one place, rather than have two flavors and tests to make sure they stay the same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, ok, this is what I meant in https://github.com/MaterializeInc/materialize/pull/33500/files#r2318981605
If you deem this refactoring worthwhile, then I'll do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, see the latest commit.
memoize_expr
06ee435
to
25e178c
Compare
25e178c
to
a7545e2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code changes look great, thank you!
Thanks for the review! (The Nightly fail looks unrelated.) |
Fixes https://github.com/MaterializeInc/database-issues/issues/9643 by making
memoize_expr
(which is called fromextract_common
) not go into temporal filters.Note that
inline_expressions
solves this problem when full MFP optimization is run, butextract_common
only runsmemoize_expressions
.The PR adds a regression test in
temporal.slt
. (temporal.td
also tests various temporal filters.) Additionally, the requirement that the non-mz_now side of a temporal filter should still be processed bymemoize_expressions
is tested by the existing EXPLAINs infilter-pushdown.slt
.The PR also makes the existing tests slightly stronger by putting actual data into one of the existing tables, and actually querying the existing materialized views to check for evaluation errors, in addition to planning errors.
Nightly (subset): https://buildkite.com/materialize/nightly/builds/13060
Edit: I've thought about feature flagging this change, but unfortunately the optimizer flags are not wired to this part of the code. But I think this is a low-risk change.
New Nightly run: https://buildkite.com/materialize/nightly/builds/13227
Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$T
mapping (possibly in a backwards-incompatible way), then it is tagged with aT-proto
label.