deno test --coverage uses a lot of RAM

[harvestcore]

Hello y'all.

I've been lately noticing some weird "Container 'Build' exceeded memory limit" errors in one of the (Bitbucket) pipelines that I've configured for my Deno app. This basically means that the process I'm executing is using too much RAM (the pipeline is configured with 8GBs of RAM).

What the pipeline does is execute the unit tests with the coverage enabled, and took me a while to notice that the issue is that as soon as the pipeline executes the main command Deno starts using A LOT of memory. I never noticed this issue when running the test+coverage in my personal computer as it has 32GBs of RAM.

I can understand that running the tests with coverage enabled is obviously going to use more memory, but I didn't expect it to be that much. Here are some details:

• Deno setup:
  • deno 1.35.2 (release, aarch64-apple-darwin)
  • v8 11.6.189.7
  • typescript 5.1.6
• 660 unit tests.
• ~20k lines of TS code.
• Coverage enabled: Almost 6GB of RAM.
• Coverage disabled: Less than 500MB.

My question is: Is this memory usage expected? Is there a way I can lower it?

[bartlomieju]

Thanks for the report. A short gist of what's happening when coverage is enabled:

1. Each test file is run in a separate worker
2. Worker start collecting coverage by instrumenting V8 to collect the coverage data
3. Once the tests in a particular file finish the coverage is "taken", stringified to JSON, formatted and written to a file
4. The process repeats for each file.

We really don't have much control in terms of what V8 is doing here - from a quick glance we are not holding the the data produced for more than necessary. It would be beneficial to do some baseline benchmarking in terms of how much memory is used for an empty test file.

[harvestcore]

Hi @bartlomieju, thanks for the quick response and sorry for my late reply. Thank you again for the information about what Deno does under the hood when coverage is enabled.

In the end we've decided to disabled the coverage for now. And we're splitting the original coverage report into two, to make it lighter.

In response to your suggestion...

It would be beneficial to do some baseline benchmarking in terms of how much memory is used for an empty test file.

I've created a simple test project to measure this. It contains 1000 empty tests (across 10 test files). First results show about ~65MB of memory usage. This means ~6.5MB per file, and ~0.065MB per test. When adding a simple expect(1).toEqual(1) in every test the memory usage is ~85MB total, ~8.5MB/file, and ~0.085MB/test.

I hope those numbers help in some way 🙂. Personally I think that the memory usage of those tests that do nothing and have no dependencies is a bit high, so I can understand that when testing a large application the memory usage is so high.