Skip to content

Option to treat timeouts as non-failure #1291

@Robbepop

Description

@Robbepop

I am currently using nextest via miri in Wasmi to run the entire Wasm spec testsuite (which is huge) nightly. Since miri execution with all of its checks is rather slow and since we use default GitHub action runners most tests timeout. Locally I ran the entire testsuite via miri with a timeout of 4h per test case and even then I had some timeouts, so that's how bad it really is. However, this is not as bad as it sounds since even timed out tests produced a ton of checks and validated their input until their timeout.

So what would be nice is to make it possible via config to treat timed out tests as non-failures since I only really want to be notified by this CRON miri run if it actually found an actual bug.

Is this maybe even already possible? In case you do not want to implement this feature what would be the best way to handling this on my side?

Link to the cron job if that is of interest: https://github.com/paritytech/wasmi/actions/runs/7771389011/job/21192633974

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions