Skip to content

Improve Azure Monitor scalability and performance #38624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
zmoog opened this issue Mar 26, 2024 · 1 comment · May be fixed by elastic/integrations#13783
Open

Improve Azure Monitor scalability and performance #38624

zmoog opened this issue Mar 26, 2024 · 1 comment · May be fixed by elastic/integrations#13783
Assignees
@zmoog @MichaelKatsoulis
Labels
enhancement Team:obs-ds-hosted-services Label for the Observability Hosted Services team

Comments

@zmoog
Copy link
Contributor

zmoog commented Mar 26, 2024

The Azure Monitor metricset can collect metrics for Azure resources such as VMs, database accounts, and more.

However, the current implementation limits how many Azure resources it can collect metrics for. As of 8.13, the limit is ~120 for PT1M metrics and ~400 for PT5M metrics. Beyond these limits, the metricset may only sometimes be able to collect values every 60s and 300s as necessary.

We plan to improve the performance and scalability of the metricset by introducing two changes:

  • Refresh resource list and metrics definitions asynchronously.
  • Collect metric values using the getBatch API.

Relates to: elastic/integrations#8575
@zmoog zmoog self-assigned this Mar 26, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 26, 2024
@axw axw added the Team:obs-ds-hosted-services Label for the Observability Hosted Services team label Mar 27, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 27, 2024
@MichaelKatsoulis MichaelKatsoulis mentioned this issue Nov 26, 2024
Merged
6 tasks
@MichaelKatsoulis MichaelKatsoulis mentioned this issue Feb 3, 2025
Closed
@mergify mergify bot mentioned this issue Apr 15, 2025
Merged
6 tasks
@MichaelKatsoulis MichaelKatsoulis linked a pull request May 5, 2025 that will close this issue
Open
5 tasks
This was referenced May 7, 2025
Draft
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zmoog zmoog

@MichaelKatsoulis MichaelKatsoulis

Labels
enhancement Team:obs-ds-hosted-services Label for the Observability Hosted Services team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add enable_batch_api option in azure resource metrics MichaelKatsoulis/integrations
4 participants
@zmoog @axw @elasticmachine @MichaelKatsoulis