Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include information on whether a query is using SSQE or MSQE in QueryLogger #14938

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

yashmayya
Copy link
Collaborator

  • Currently, the only way to distinguish between SSQE and MSQE queries via query logs in Pinot brokers is to check whether serverStats is empty (MSQE) or not (SSQE).
  • This isn't reliable or convenient and we should explicitly indicate which query engine the query is executing with for easier log analysis.

@yashmayya yashmayya added enhancement observability multi-stage Related to the multi-stage query engine labels Jan 29, 2025
@yashmayya yashmayya requested a review from gortiz January 29, 2025 16:46
@@ -255,6 +257,12 @@ void doFormat(StringBuilder builder, QueryLogger logger, QueryLogParams params)
builder.append(CommonConstants.UNKNOWN);
}
}
},
IS_MULTI_STAGE_QUERY_ENGINE("isMultiStageQueryEngine") {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could call this queryEngine with possible options being singleStage / multiStage so that it is open to future extensions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that idea

@codecov-commenter
Copy link

codecov-commenter commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 80.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 63.70%. Comparing base (59551e4) to head (33d4dad).
Report is 1648 commits behind head on master.

Files with missing lines Patch % Lines
...sthandler/BaseSingleStageBrokerRequestHandler.java 0.00% 2 Missing ⚠️
...requesthandler/MultiStageBrokerRequestHandler.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14938      +/-   ##
============================================
+ Coverage     61.75%   63.70%   +1.94%     
- Complexity      207     1472    +1265     
============================================
  Files          2436     2710     +274     
  Lines        133233   151931   +18698     
  Branches      20636    23462    +2826     
============================================
+ Hits          82274    96782   +14508     
- Misses        44911    47881    +2970     
- Partials       6048     7268    +1220     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.68% <80.00%> (+1.97%) ⬆️
java-21 63.57% <80.00%> (+1.95%) ⬆️
skip-bytebuffers-false 63.69% <80.00%> (+1.95%) ⬆️
skip-bytebuffers-true 63.55% <80.00%> (+35.83%) ⬆️
temurin 63.70% <80.00%> (+1.94%) ⬆️
unittests 63.69% <80.00%> (+1.94%) ⬆️
unittests1 56.22% <ø> (+9.33%) ⬆️
unittests2 34.01% <80.00%> (+6.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yashmayya yashmayya requested a review from gortiz January 30, 2025 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement multi-stage Related to the multi-stage query engine observability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants