push down filter: consider already pushed filters #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
askalt
force-pushed
the
askalt/push-already-pushed-filters
branch
from
399cf94 to
c21fd06
Compare
If optimizer runs several times it can try to push filters into table provider several times. But, `supports_filters_pushdown` may be context-dependent. For example, it may support any filter like `column=value` but not the both at the same time. Consider the following optimizer run: 1. Try to push `a = 1, b = 1`. `supports_filters_pushdown` returns [Exact, Inexact] Ok, will remember that `a=1` is pushed and make a filter node for `b=1`. ... Another optimization iteration. 2. Try to push `b = 1`. `supports_filters_pushdown` returns [Exact]. Of course, the table provider can't remember all pushed filters. So, there is nothing to left but to answer: "Exact". Now, the optimizer thinks that conjunction `a = 1 AND b = 1` is supported exactly, but it's not. To prevent this problem, this patch adds already pushed filters to the `supports_filters_pushdown` call.
askalt
force-pushed
the
askalt/push-already-pushed-filters
branch
from
c21fd06 to
3b3b5d1
Compare
psergee
approved these changes
Jun 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If optimizer runs several times it can try to push filters into table provider several times. But,
supports_filters_pushdownmay be context-dependent. For example, it may support any filter likecolumn=valuebut not the both at the same time.Consider the following optimizer run:
a = 1, b = 1.supports_filters_pushdownreturns [Exact, Inexact] Ok, will remember thata=1is pushed and make a filter node forb=1....
Another optimization iteration.
b = 1.supports_filters_pushdownreturns [Exact]. Of course, the table provider can't remember all pushed filters. So, there is nothing to left but to answer: "Exact". Now, the optimizer thinks that conjunctiona = 1 AND b = 1is supported exactly, but it's not.To prevent this problem, this patch adds already pushed filters to the
supports_filters_pushdowncall.