test_runner: allow to force exit after all tests finished

[rluvaton]

What is the problem this feature will solve?

if you have some resource that you don't control blocks the process from exiting (socket/timer/etc) the tests will never finish

having timeout in the run options marking the test as failed

What is the feature you are proposing to solve the problem?

allow to specify force exit option that after all tests finished will exit

similar solutions in other frameworks:

• mocha has --exit flag
• jest has --forceExit

What alternatives have you considered?

No response

[rluvaton]

cc @nodejs/test_runner

[aduh95]

Do we need to do anything? Wouldn't a process.exit() call in after already does what you want?

[cjihrig]

I don't quite understand the use case.

if you have some resource that you don't control

It's your test suite, so in theory you have some level of control over all of the resources. It sounds like you just aren't cleaning up after yourself properly.

[MoLow]

in node:test there is no such event as "after all tests finished".

this is totally valid:

await setTimeout(10 * 60 * 1000) // sleep for ten minutes
test('run a test after ten minutes', () => {});

and there is no way to know a test is going to be enqueued besides waiting for event loop to drain using process.on('beforeExit') - that is what we do already.

[rluvaton]

Do we need to do anything? Wouldn't a process.exit() call in after already does what you want?

process.exit will mess with the reporting as the report will not have enough time to reach the parent process (waiting for a second and then exiting solves this but I don't think it's a solution)

[rluvaton]

I don't quite understand the use case.

if you have some resource that you don't control

It's your test suite, so in theory you have some level of control over all of the resources. It sounds like you just aren't cleaning up after yourself properly.

Unfortunately, the use case is some dependency of dependency that I'm using is not cleaning up...

[atlowChemi]

Unfortunately, the use case is some dependency of dependency that I'm using is not cleaning up...

@rluvaton that sounds to me like a bug within the dependency, which should be fixed there...

In addition, as Moshe said, the test runner can't tell if all tests completed, as we do not build the test tree before executing the tests, which means adding such an implementation would also make executing the tests that much slower...

[benjamingr]

I think this is reasonable and test-runners typically implement this because it's common enough, I'm also not sure we can implement this reasonably with our current design.

We can however (with a flag) exit when no more tests are enqueued within a microtick after all already enqueued tests were run which should work well in practice maybe?

[cjihrig]

I disagree. I don't think we should implement this, if for no other reason than I don't think it's worth another CLI flag. You're literally saying you have no control over your own application if you can't make it stop.

[MoLow]

another option is to add on the TestContext a getter with the pending tests count, this wat on the after/afterEach hook it will be possible to run cleanup code depending on that getter

[cjihrig]

I don't mind that idea, but if I understand #49925 (comment) correctly, the problem is that they are not able to perform any cleanup.

(I'm curious to hear more about this by the way because if the issue is that a handle is holding the event loop open, you should be able to use something like process._getActiveHandles() to find it and likely force close it. It's not ideal, but it sounds like neither is the current situation.)