StartStopTests.CreateStartAndStopAspireProject is flaky — times out polling docker network

[davidfowl]

Description

Aspire.Cli.EndToEnd.Tests.StartStopTests.CreateStartAndStopAspireProject is timing out in CI, polling docker network ls 480+ times without the network disappearing.

Failure Details

The test runs aspire stop and then repeatedly checks docker network ls --format json | grep -i -- 'StarterApp_...' | wc -l waiting for the network to be removed. It never reaches 0 and times out after ~9 minutes.

From CI run 24295652186:

[480 OK] $ docker network ls --format json | grep -i -- 'StarterApp_14a089' | wc -l
1
[481 OK] $
Hex1b.Automation.Hex1bAutomationException : Step 1932 of 1932 failed — WaitUntil(...)

Recent Change

The most recent change to this test file was #16006 (Fix issue 15806 — aspire stop is not cleaning up application containers) by @karolz-ms, merged April 10.

This likely changed the container cleanup behavior for aspire stop, and the test's network cleanup polling may no longer be compatible with the new behavior.

Reproduction

Observed in CI on ubuntu-latest. Likely contention-sensitive (only fails alongside other tests).

/cc @karolz-ms

[davidfowl]

Another occurrence: https://github.com/microsoft/aspire/actions/runs/24314128290/job/70989001903 (PR #16093, unrelated to changes — docker network cleanup timeout at attempt 468)

[karolz-ms]

It may be a flaky test, but it also might be a real issue (that the cleanup is not always happening as it should).