Summary
The CLI E2E tests have grown organically and would benefit from a unified helper layer that centralizes common patterns, error handling, and diagnostics.
Current state
CliE2EAutomatorHelpers.cs has AspireStartAsync/AspireStopAsync with good error handling (log dumping, dashboard validation)
- Individual tests duplicate patterns for
aspire init, aspire add, aspire restore, aspire run + Ctrl+C
- Some tests use
CreateDockerTestTerminal, others use CreateTestTerminal — different environments with different capabilities
- Error diagnostics (log capture, JSON output inspection) are inline in
AspireStartAsync but not available for other commands
- Resource graph assertions are ad-hoc (e.g., checking file existence, grepping output)
Proposed improvements
1. Reusable resource graph assertions
- Parse
aspire describe --format json output into typed objects
- Link the shared
ResourceJson model (src/Shared/Model/Serialization/ResourceJson.cs) into the E2E test project
- Build assertion helpers:
AssertResourcesExistAsync, AssertResourceHasRelationshipAsync, AssertResourceStateAsync, etc.
- Any E2E test that cares about resource materialization can use these
2. Centralized error handling and diagnostics
- Extract the log-dumping logic from
AspireStartAsync into a reusable helper that any command can use on failure
- Standardize how tests capture and surface CLI logs, detach logs, and JSON output on failure
- Make it easy to add diagnostic capture to any
aspire command, not just start
3. Consistent terminal setup patterns
- Document when to use
CreateDockerTestTerminal vs CreateTestTerminal
- Consider a unified factory that picks the right terminal based on what the test needs (Docker socket, Node.js, etc.)
4. Verbose logging control
- Add a helper to enable/disable verbose CLI logging (
--log-level) consistently across tests
- Centralize log level configuration so debugging a failing E2E test is as simple as flipping a flag
Context
This came up during PR #15901 where we needed to assert un-awaited promise chains actually created resources at runtime. We added AssertResourcesExistAsync as a first step, but it uses node to parse JSON in the container rather than the typed ResourceJson model. A proper solution would make the shared model available to E2E tests.
Summary
The CLI E2E tests have grown organically and would benefit from a unified helper layer that centralizes common patterns, error handling, and diagnostics.
Current state
CliE2EAutomatorHelpers.cshasAspireStartAsync/AspireStopAsyncwith good error handling (log dumping, dashboard validation)aspire init,aspire add,aspire restore,aspire run+ Ctrl+CCreateDockerTestTerminal, others useCreateTestTerminal— different environments with different capabilitiesAspireStartAsyncbut not available for other commandsProposed improvements
1. Reusable resource graph assertions
aspire describe --format jsonoutput into typed objectsResourceJsonmodel (src/Shared/Model/Serialization/ResourceJson.cs) into the E2E test projectAssertResourcesExistAsync,AssertResourceHasRelationshipAsync,AssertResourceStateAsync, etc.2. Centralized error handling and diagnostics
AspireStartAsyncinto a reusable helper that any command can use on failureaspirecommand, not juststart3. Consistent terminal setup patterns
CreateDockerTestTerminalvsCreateTestTerminal4. Verbose logging control
--log-level) consistently across testsContext
This came up during PR #15901 where we needed to assert un-awaited promise chains actually created resources at runtime. We added
AssertResourcesExistAsyncas a first step, but it usesnodeto parse JSON in the container rather than the typedResourceJsonmodel. A proper solution would make the shared model available to E2E tests.