Update outerloop workflow #9269
Open
+291
−262
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
needs-area-label
There was a problem hiding this comment.
Pull Request Overview
This PR updates the outerloop workflow and related test infrastructure to align with the interloop runsheet-based process and to explicitly opt-in to parameter randomisation. Key changes include:
- Adding an explicit parameter (enableParameterRandomisation) for tests in various factories and endpoints.
- Updating the QuarantinedTestRunsheetBuilder.targets to include and process pre-commands.
- Tweaking conditions in the outerloop workflow YAML to adjust log upload behavior.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/Aspire.Playground.Tests/ProjectSpecificTests.cs | Updated test host creation calls with explicit enableParameterRandomisation parameter. |
| tests/Aspire.Playground.Tests/Infrastructure/DistributedApplicationTestFactory.cs | Modified factory signature to include parameter randomisation configuration. |
| tests/Aspire.Playground.Tests/AppHostTests.cs | Adjusted test endpoints initialization to include the new parameter. |
| eng/QuarantinedTestRunsheetBuilder/QuarantinedTestRunsheetBuilder.targets | Added pre-command file name properties and updated commands for runsheet generation. |
| .github/workflows/tests-outerloop.yml | Tweaked workflow steps, including log upload conditions and added new summary generation step. |
Comments suppressed due to low confidence (1)
.github/workflows/tests-outerloop.yml:36
- [nitpick] Changing the log upload condition from 'always()' to 'failure()' may be intentional, but please verify that logs should only be uploaded on failures per the new workflow requirements.
if: failure()
eng/QuarantinedTestRunsheetBuilder/QuarantinedTestRunsheetBuilder.targets
Show resolved
Hide resolved
|
Can we trigger a outerloop run for this branch? |
|
Changes look good to me. I just want to confirm with a Outerloop run that my existing tool can still work. |
This comment was marked as outdated.
This comment was marked as outdated.
danmoseley
added
area-engineering-systems
* Move known flaky/unhealthy tests into own theory. This allows other tests can be executed in the interloop run, and the flaky to be constrained to the outerloop. * Disable SqlServerEndToEnd test, see #9274 * Add collections to serialise test execution
|
Playground tests are still failing. Do we need the playground test changes in this PR? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
tests can be executed in the interloop run, and the flaky to be constrained to the outerloop.
ParameterEndToEnd(SqlServerEndToEnd_AppHost)crashes in outerloop #9274