Test enable groups as specified by PR labels #2357
Conversation
This was mostly for use in the `main` building case, because otherwise it's maybe a bit too easy to forget to update this file when adding an enable group
|
When I've added this sort of thing in the past, it wouldn't trigger the expanded tests on the first push (since there's no label there), and then you have to add the label then push something to get the expanded testing. |
|
I did set it to trigger on 'labelled', so hopefully that shouldn't be a problem? Once I get these tests to pass we can play with it. |
|
Cool, I added the PyPy tag and the tests reran. I'll update this table when it finishes:
|
|
Looks like it's working based on the above table! The remaining question is what to do with the other CIs. Perhaps the right approach is to run everything on |
|
That's done. This is ready to review. |
|
Should we merge #1538 first, then add graalpy here? Also, I'd like to remove the deprecated PyPy's or put then behind a separate enable group. The original reason to add an enable group was to avoid depreciated PyPy's, but since it enables both current and deprecated PyPy's, it's not helpful for the original use case. |
Yeah, that would make sense to me.
Sounds good to me! |
Don't run all enable groups on PRs, but allow them to be enabled using PR labels.
This is mostly to keep the PR builds fast. As noted in #1538 (comment) , extra enable groups affect the CI times a lot.
The only CI affected by the labels is Github Actions. The others just run the default set, which probably isn't ideal, but I can't think of another way to do it!