Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return JSON instead of str in the broker debug api /debug/serverRoutingStats #15018

Merged
merged 2 commits into from
Feb 12, 2025

Conversation

vrajat
Copy link
Collaborator

@vrajat vrajat commented Feb 10, 2025

Fix for #15017

The o/p of the api is not easily machine readable. An example o/p is:

(Server=NumInFlightRequests,NumInFlightRequestsEMA,LatencyEMA,Score);Server_100.81.71.117_7050=0,6.349956168651998E-148,2.5324251579231667E-148,0.0;Server_100.81.71.117_7051=0,6.150450664351991E-147,5.482000636051586E-147,0.0;Server_100.81.71.117_7052=0,6.861196743124505E-148,2.526062961096265E-148,0.0;Server_100.81.71.117_7053=0,3.343930442767001E-147,5.466877618135145E-145,0.0

Instead return a json since its more easily analyzed using scripts. For example, the following json o/p of the same data:

{
  "Server_100.81.71.117_7050": {
    "latencyEMA": 2.6305286728948865,
    "numInFlightRequests": 10,
    "inFlightRequestsEMA": 10.83252545140485,
    "hybridScore": 23783.118690453754
  },
  "Server_100.81.71.117_7051": {
    "latencyEMA": 2.546575260838141,
    "numInFlightRequests": 10,
    "inFlightRequestsEMA": 10.832524167798587,
    "hybridScore": 23024.07539269138
  },
  "Server_100.81.71.117_7052": {
    "latencyEMA": 2.8777880660250883,
    "numInFlightRequests": 10,
    "inFlightRequestsEMA": 10.83252545140485,
    "hybridScore": 26018.638703878878
  },
  "Server_100.81.71.117_7053": {
    "latencyEMA": 1603.1813971374418,
    "numInFlightRequests": 0,
    "inFlightRequestsEMA": 18.5040581516264,
    "hybridScore": 10157424.86651867
  }
}

Can be processed by shell scripts to create csv files and plot them.

This change is backward-incompatible. However this is a debug API.

@vrajat vrajat requested a review from vvivekiyer February 10, 2025 09:38
@vrajat vrajat added backward-incompat Referenced by PRs that introduce or fix backward compat issues observability labels Feb 10, 2025
@codecov-commenter
Copy link

codecov-commenter commented Feb 10, 2025

Codecov Report

Attention: Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 63.68%. Comparing base (59551e4) to head (c3dee31).
Report is 1692 commits behind head on master.

Files with missing lines Patch % Lines
...e/pinot/broker/api/resources/PinotBrokerDebug.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15018      +/-   ##
============================================
+ Coverage     61.75%   63.68%   +1.93%     
- Complexity      207     1483    +1276     
============================================
  Files          2436     2727     +291     
  Lines        133233   152460   +19227     
  Branches      20636    23565    +2929     
============================================
+ Hits          82274    97096   +14822     
- Misses        44911    48051    +3140     
- Partials       6048     7313    +1265     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.64% <25.00%> (+1.93%) ⬆️
java-21 63.57% <25.00%> (+1.95%) ⬆️
skip-bytebuffers-false 63.65% <25.00%> (+1.90%) ⬆️
skip-bytebuffers-true 63.54% <25.00%> (+35.82%) ⬆️
temurin 63.68% <25.00%> (+1.93%) ⬆️
unittests 63.68% <25.00%> (+1.93%) ⬆️
unittests1 56.23% <100.00%> (+9.34%) ⬆️
unittests2 34.00% <0.00%> (+6.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vrajat
Copy link
Collaborator Author

vrajat commented Feb 11, 2025

cc @somandal

@@ -198,6 +198,10 @@ private void updateStatsUponResponseArrival(String serverInstanceId, long latenc
}
}

public Map<String, ServerRoutingStatsEntry> getServerRoutingStats() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add tests for this in ServerRoutingStatsManagerTest to validate that it looks as expected?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -198,6 +198,10 @@ private void updateStatsUponResponseArrival(String serverInstanceId, long latenc
}
}

public Map<String, ServerRoutingStatsEntry> getServerRoutingStats() {
return _serverQueryStatsMap;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us check for _isEnabled here? _serverQueryStatsMap would be null if config is disabled.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait - I checked for isEnabled in the PinotBrokerDebug file. Is that OK ? Right now, it is the caller's responsibility to check for isEnabled and null.

@vvivekiyer vvivekiyer merged commit 6f726bb into apache:master Feb 12, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backward-incompat Referenced by PRs that introduce or fix backward compat issues observability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants