Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add system stats to telemetry #686

Open
Mirko-von-Leipzig opened this issue Feb 12, 2025 · 0 comments
Open

Add system stats to telemetry #686

Mirko-von-Leipzig opened this issue Feb 12, 2025 · 0 comments
Labels
telemetry Relates to tracing and open-telemetry

Comments

@Mirko-von-Leipzig
Copy link
Contributor

We probably want to track at least the following:

  • storage disk space used and available
  • memory use and remaining
  • cpu usage and info

Ideal would also be to list our own process's usage specifically, including

  • database size
  • database WAL size
  • block storage used
  • memory used
  • cpu times etc

Remember to check out the open-telemetry naming specification for system stats (yes, the page is about metrics, but we're injecting metrics into wide-events so let's stick with the conventions).

Implementation

Two options I can think of:

  1. Background thread that periodically posts these stats as an event
  2. Inject them into every span we emit

I'm assuming getting this information requires syscalls, and as such its not reasonable to do (2) in the actual span processor. Instead we likely still want a background thread to periodically update the values.

https://docs.rs/sysinfo/latest/sysinfo/ is a popular option I found, but maybe do some more research first. We want something low-cost. I also see that macOs could be problematic due to security permissions - we'll have to check that out as well.

@Mirko-von-Leipzig Mirko-von-Leipzig added the telemetry Relates to tracing and open-telemetry label Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
telemetry Relates to tracing and open-telemetry
Projects
None yet
Development

No branches or pull requests

1 participant