question about memory usage

[frizwi]

Summary

Hi, exactly how is the memory usage tracked? Presently its grossly under reporting but that's probably due to the shell script running jobs in the background. Is there a way to JSON output the pid of the actual job so that its more reflective of the actual usage?

In case you're interested, it ends up being a Docker container that's invoked

Thanks!
-Farhan

Steps to reproduce the problem

Your Setup

Operating system and version?

Node.js version?

Cronicle software version?

Are you using a multi-server setup, or just a single server?

Are you using the filesystem as back-end storage, or S3/Couchbase?

Can you reproduce the crash consistently?

Log Excerpts

[audetto]

I have a very similar issue.
I run it inside a container which has very strict memory limits.

In particular the current memory usage is available in a file /sys/fs/cgroup/memory/memory.usage_in_bytes and one can get limit_in_bytes too.

How is cronicle determining this?

Problem is that if it goes above, the container is rebooted quickly.
Can we communicate to cronicle how to get a better snapshot of memory usage?

[jhuckaby]

Cronicle tracks memory per job by using /bin/ps -eo "ppid pid %cpu rss" output and building a tree of PID / PPID relationships. All of the job process rss numbers are added up (resident memory). It does not track things like virtual memory.

https://github.com/jhuckaby/Cronicle/blob/master/lib/job.js#L1701-L1830

If you are launching jobs that then spawn their own "detached" processes that don't have a correct PPID (parent PID) set, or if your processes are using virtual rather than resident memory, then it may not track memory correctly.

🤷🏻‍♂️

[audetto]

Understood thanks.