Prevent excessively long regression test runs

[jonmaddock]

Recently (#3565) I encountered a regression test (ST) that ran for well over an hour, with >1000 iterations. We should add a limit to prevent excessively long regression runs that aren't even going to converge.

Additionally, I found that cancelling the workflow didn't work: there was no way of stopping the job in the UI. I had to force-cancel using the gh CLI.