Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add requests rate, message rate to the linear regression model to predict CPU #3

Open
becketqin opened this issue Aug 27, 2017 · 2 comments
Labels
correctness A condition affecting the proper functionality.

Comments

@becketqin
Copy link
Contributor

Currently we only use bytes in and bytes out as parameters to predict the CPU utilization in our linear regression model, which may not work very well. We should include request and message rate into the linear regression model.

@igorcalabria
Copy link

From our experience, request rate(especially produce requests) is a great metric to correlate with cpu usage. This is important because if you have any jobs(mapreduce, spark, etc) that write to kafka you'll have a massive number of messages and a small number of requests because of batching. This kind of workload barely impacts our broker's CPU.

On the other hand, web applications that don't have the same ability to batch messages causes a huge impact on performance with fewer messages

@becketqin
Copy link
Contributor Author

@igorcalabria Thanks for the information. That is also what we witnessed. I am currently working on the experiment. Actually the PartitionMetricSamples and BrokerMetricSamples has already contain the RequestRate. We just need to put them into the cluster model so it can be used during the optimization.

sudoa pushed a commit to sudoa/cruise-control that referenced this issue Nov 22, 2018
@efeg efeg added the correctness A condition affecting the proper functionality. label May 21, 2019
viktorsomogyi added a commit to viktorsomogyi/cruise-control that referenced this issue Nov 3, 2021
This PR synchronizes the vertx start method so the start method will return only when the verticle has been deployed and is running.
viktorsomogyi added a commit to viktorsomogyi/cruise-control that referenced this issue Dec 14, 2021
This PR synchronizes the vertx start method so the start method will return only when the verticle has been deployed and is running.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
correctness A condition affecting the proper functionality.
Projects
None yet
Development

No branches or pull requests

3 participants