Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow optional second argument denoting timezone for datetime extraction functions in the multi-stage engine #14998

Merged
merged 1 commit into from
Feb 5, 2025

Conversation

yashmayya
Copy link
Collaborator

  • The date time extract functions in Pinot like YEAR, DAY_OF_YEAR, DAY_OF_WEEK etc. support an optional second argument denoting the timezone for the timestamp (both the transform and scalar function implementations support this).
  • However, we use the standard operator definitions from Calcite in the multi-stage engine that only support a single operand (the datetime type). This leads to errors like Invalid number of arguments to function 'DAYOFWEEK'. Was expecting 1 argument on attempting to use the function like DAYOFWEEK(ts, 'UTC').
  • This patch fixes the issue by removing the standard operators from the operator table and instead defining our own custom operators with the appropriate operand type checkers (the return type inference is kept the same as the one from the standard operators).

@yashmayya yashmayya added bugfix multi-stage Related to the multi-stage query engine labels Feb 5, 2025
@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2025

Codecov Report

Attention: Patch coverage is 54.16667% with 11 lines in your changes missing coverage. Please review.

Project coverage is 63.67%. Comparing base (59551e4) to head (b49fd3f).
Report is 1673 commits behind head on master.

Files with missing lines Patch % Lines
...e/pinot/common/function/TransformFunctionType.java 54.16% 11 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14998      +/-   ##
============================================
+ Coverage     61.75%   63.67%   +1.92%     
- Complexity      207     1483    +1276     
============================================
  Files          2436     2713     +277     
  Lines        133233   152183   +18950     
  Branches      20636    23526    +2890     
============================================
+ Hits          82274    96900   +14626     
- Misses        44911    47980    +3069     
- Partials       6048     7303    +1255     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.64% <54.16%> (+1.93%) ⬆️
java-21 63.56% <54.16%> (+1.93%) ⬆️
skip-bytebuffers-false 63.66% <54.16%> (+1.91%) ⬆️
skip-bytebuffers-true 63.53% <54.16%> (+35.80%) ⬆️
temurin 63.67% <54.16%> (+1.92%) ⬆️
unittests 63.66% <54.16%> (+1.92%) ⬆️
unittests1 56.20% <54.16%> (+9.31%) ⬆️
unittests2 34.03% <50.00%> (+6.29%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit e2575fc into apache:master Feb 5, 2025
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix multi-stage Related to the multi-stage query engine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants