Skip to content

Conversation

@fgiannar
Copy link
Contributor

This PR upgrades the Docker env to use Node 20.

To test

  1. Apply PR
  2. Rebuild Docker env
  3. Run Jetmon
  4. Make sure it works.

@chrisbliss18
Copy link
Contributor

I ran this branch with various configs throughout the day. With my original config, I saw many warnings like the following:

node[3404028]: pthread_create: Resource temporarily unavailable

and

/usr/local/bin/node[3404080]: ../src/node_platform.cc:68:std::unique_ptr<long unsigned int> node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion `(0) == (uv_thread_create(t.get(), start_thread, this))' failed. 
 1: 0xc9adf0 node::Abort() [/usr/local/bin/node]
 2: 0xc9ae6e  [/usr/local/bin/node]
 3: 0xd194b9 node::WorkerThreadsTaskRunner::WorkerThreadsTaskRunner(int) [/usr/local/bin/node]
 4: 0xd195dc node::NodePlatform::NodePlatform(int, v8::TracingController*, v8::PageAllocator*) [/usr/local/bin/node]
 5: 0xc548c3  [/usr/local/bin/node]
 6: 0xc55254 node::Start(int, char**) [/usr/local/bin/node]
 7: 0x7fa63a60c1ca  [/lib/x86_64-linux-gnu/libc.so.6]
 8: 0x7fa63a60c285 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
 9: 0xbb0e5e _start [/usr/local/bin/node]

I originally had a NUM_WORKERS of 800. When I dropped it down to 200, the messages no longer appeared, even after running for a few hours.

It seems that v20 has much tighter restrictions on spawned threads potentially exceeding memory limits. This is a bit of a double-edged sword for us. On the one hand, it could help prevent eating up too much memory. On the other hand, when running with a conf that resulted in spawning these warnings, the performance of the service was really poor and the metrics were very chaotic. So, we're going to have to really tune the configs tightly if we are to roll this out to production.

@fgiannar
Copy link
Contributor Author

Didn't work in production as expected. Added DO NOT MERGE label to ensure we only merge this after successfully upgrading production to Node.js v20

@fgiannar fgiannar added the janitorial Cleanup on isle 4... label Sep 14, 2023
@andrii-lysenko
Copy link

@fgiannar @chrisbliss18 I'm wondering if there's a reason we are using full Node version instead of alpine version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DO NOT MERGE janitorial Cleanup on isle 4...

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants