You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we only use bytes in and bytes out as parameters to predict the CPU utilization in our linear regression model, which may not work very well. We should include request and message rate into the linear regression model.
The text was updated successfully, but these errors were encountered:
From our experience, request rate(especially produce requests) is a great metric to correlate with cpu usage. This is important because if you have any jobs(mapreduce, spark, etc) that write to kafka you'll have a massive number of messages and a small number of requests because of batching. This kind of workload barely impacts our broker's CPU.
On the other hand, web applications that don't have the same ability to batch messages causes a huge impact on performance with fewer messages
@igorcalabria Thanks for the information. That is also what we witnessed. I am currently working on the experiment. Actually the PartitionMetricSamples and BrokerMetricSamples has already contain the RequestRate. We just need to put them into the cluster model so it can be used during the optimization.
sudoa
pushed a commit
to sudoa/cruise-control
that referenced
this issue
Nov 22, 2018
Currently we only use bytes in and bytes out as parameters to predict the CPU utilization in our linear regression model, which may not work very well. We should include request and message rate into the linear regression model.
The text was updated successfully, but these errors were encountered: