IHostApplicationLifetime StopApplication not respected in AspNetCore.Mvc.Testing

[GeeWee]

Describe the bug

I'm trying to run integration tests, with some things that call IHostApplicationLifetime.StopApplication from an IHostedService

In a "real project" this works, but it does not when using the TestServer.
Test case below

To Reproduce

namespace DefaultNamespace
{
    using System.Threading;
    using System.Threading.Tasks;
    using BetterHostedServices.Test;
    using Microsoft.AspNetCore.Builder;
    using Microsoft.AspNetCore.Hosting;
    using Microsoft.AspNetCore.Mvc.Testing;
    using Microsoft.AspNetCore.TestHost;
    using Microsoft.Extensions.DependencyInjection;
    using Microsoft.Extensions.Hosting;
    using Xunit;

    public class Test
    {
// Dummy startup that does nothing
        public class TestStartup
        {
            public void ConfigureServices(IServiceCollection services)
            {
            }

            public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
            {
            }
        }

// Hosted Service that stops the application
        public class HostedServiceThatShutsDown : IHostedService
        {
            private IHostApplicationLifetime lifetime;

            public HostedServiceThatShutsDown(IHostApplicationLifetime lifetime)
            {
                this.lifetime = lifetime;
            }

            public Task StartAsync(CancellationToken cancellationToken)
            {
                this.lifetime.StopApplication();
                return Task.CompletedTask;
            }

            public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;
        }

// WebApplicationFactory with a few modifications so it can stand-alone
        public class TestWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup> where TStartup : class
        {
            protected override void ConfigureWebHost(IWebHostBuilder builder)
            {
                builder.UseSolutionRelativeContentRoot(
                    "./tests/IntegrationUtils"); // Just because I have a custom dir in my project, can disregard
            }

            //from https://stackoverflow.com/questions/61159115/asp-net-core-testing-no-method-public-static-ihostbuilder-createhostbuilders
            protected override IHostBuilder CreateHostBuilder()
            {
                var builder = Host.CreateDefaultBuilder()
                    .ConfigureWebHostDefaults(x =>
                    {
                        x.UseStartup<DummyStartup>();
                    });
                return builder;
            }
        }

        [Fact]
        public void IssueRepro()
        {
            var factory = new TestWebApplicationFactory<TestStartup>()
                .WithWebHostBuilder(b =>
                    b.ConfigureTestServices(services =>
                    {
                        services.AddHostedService<HostedServiceThatShutsDown>();
                    }));

            var client = factory.CreateClient();

            // Works fine, but it shouldn't. The createClient above should error
            client.GetAsync("/");
        }
    }
}

The IHostedService calls StopApplication. I then don't expect the GetAsync call to work then.
The logs look like this:

info: Microsoft.Hosting.Lifetime[0]
      Application is shutting down...
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /home/geewee/programming/BetterHostedServices/tests/IntegrationUtils
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://localhost/  
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 18.5779ms 404 

So it starts off by writing "Application is shutting down" and then proceeds to start the application.

Further technical details

• ASP.NET Core version: 3.1
• Include the output of dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.1.401
 Commit:    39d17847db

Runtime Environment:
 OS Name:     fedora
 OS Version:  32
 OS Platform: Linux
 RID:         fedora.32-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.401/

Host (useful for support):
  Version: 3.1.7
  Commit:  fcfdef8d6b

.NET Core SDKs installed:
  2.2.402 [/usr/share/dotnet/sdk]
  3.0.103 [/usr/share/dotnet/sdk]
  3.1.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

• The IDE (VS / VS Code/ VS4Mac) you're running on, and it's version
  Rider, but I don't think that matters. All of this is run from the Console.

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[quixoticaxis]

This, and the fact that MVC.Testing inner infrastructure only disposes the host without stopping it first, makes MVC.Testing really hard to use with any service that has lasting effects on infrastructure.