Commit a16501f
committed
fix: Normalize timestamp literals to nanoseconds in filter pushdown
Ensure all timestamp literals are converted to nanoseconds before generating
KQL and SQL filters for pushdown, regardless of their original precision
(milliseconds, microseconds, etc.).
Problem:
Presto supports multiple timestamp precisions (TIMESTAMP with millisecond
precision, TIMESTAMP_MICROSECONDS, etc.), but CLP's query engine expects
timestamps in a consistent nanosecond format. Previously, timestamp literals
were passed through without precision normalization, causing incorrect
results when different timestamp types were used in queries.
Solution:
Add timestamp normalization logic in ClpFilterToKqlConverter:
- tryEnsureNanosecondTimestamp(): Detects TIMESTAMP and TIMESTAMP_MICROSECONDS types
- ensureNanosecondTimestamp(): Converts timestamp values to nanoseconds using:
1. Extract epoch seconds via TimestampType.getEpochSecond()
2. Extract nanosecond fraction via TimestampType.getNanos()
3. Convert to total nanoseconds: SECONDS.toNanos(seconds) + nanosFraction
Applied to:
- BETWEEN operator: Both lower and upper bounds
- Comparison operators: All timestamp literal values (=, !=, <, >, <=, >=)
This ensures consistent timestamp representation in both KQL filters (sent to
CLP engine) and SQL filters (used for metadata filtering), regardless of the
timestamp precision used in the original Presto query.
Example:
Query: WHERE timestamp BETWEEN TIMESTAMP '2025-01-01' AND TIMESTAMP '2025-01-02'
Before: Literal values passed as milliseconds or microseconds
After: Both bounds normalized to nanosecond representation1 parent 95ed6c5 commit a16501f
File tree
1 file changed
+27
-2
lines changed- presto-clp/src/main/java/com/facebook/presto/plugin/clp/optimization
1 file changed
+27
-2
lines changedLines changed: 27 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| |||
259 | 261 | | |
260 | 262 | | |
261 | 263 | | |
262 | | - | |
263 | | - | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
264 | 268 | | |
265 | 269 | | |
266 | 270 | | |
| |||
443 | 447 | | |
444 | 448 | | |
445 | 449 | | |
| 450 | + | |
446 | 451 | | |
447 | 452 | | |
448 | 453 | | |
| |||
939 | 944 | | |
940 | 945 | | |
941 | 946 | | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
942 | 967 | | |
943 | 968 | | |
944 | 969 | | |
| |||
0 commit comments