Improvements to Copilot SDK + Hosted Agent sample

[SteveSandersonMS]

As discussed with Lakshmi, Glenn, and others, here's a proposal for some ways to improve the samples for Copilot SDK + Hosted Agents.

Currently there's only a sample for that in Python, so here I'm giving an example in C# that I hope can be added, plus the existing Python sample can be similarly updated.

Note the // IMPROVEMENT and // SUGGESTION comments. These shouldn't end up in the final sample - I'm putting those here to call out ways the code here differs from the existing Python sample for clarity.

// Copyright (c) Microsoft. All rights reserved.
#pragma warning disable GHCP001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

/*
 * Hello World — Copilot SDK for C#
 *
 * This sample demonstrates integration with GitHub Copilot SDK, backed by a Foundry model.
 * 
 * Copilot SDK provides a state-of-the-art agent harness - the same agent that powers
 * Copilot CLI. For more information on options and usage, see https://github.com/github/copilot-sdk
 *
 * Required environment variables:
 *   FOUNDRY_PROJECT_ENDPOINT  — Foundry project endpoint (auto-injected in hosted containers)
 *   AZURE_AI_MODEL_DEPLOYMENT_NAME     — Model deployment name (declared in agent.manifest.yaml)
 *
 * Usage:
 *   dotnet run
 *
 *   # Turn 1 — start a new conversation:
 *   curl -sS -N -X POST "http://localhost:8088/invocations?agent_session_id=chat-001" \
 *     -H "Content-Type: application/json" \
 *     -d '{"message": "What is Microsoft Foundry?"}'
 *
 *   # Turn 2 — continue the same conversation:
 *   curl -sS -N -X POST "http://localhost:8088/invocations?agent_session_id=chat-001" \
 *     -H "Content-Type: application/json" \
 *     -d '{"message": "What hosted agent options does it offer?"}'
 */

using Azure.AI.AgentServer.Invocations;
using Azure.Identity;
using GitHub.Copilot;
using System.Text.Json;

// One-liner startup — wires up Kestrel on port 8088, OpenTelemetry, health probes,
// and the Invocations API endpoints. Telemetry is configured automatically:
// when APPLICATIONINSIGHTS_CONNECTION_STRING is set, traces and logs are sent to
// Application Insights with no extra code.
InvocationsServer.Run<HelloWorldHandler>(configure: builder =>
{
    if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING")))
        Console.Error.WriteLine(
            "[WARNING] APPLICATIONINSIGHTS_CONNECTION_STRING not set — traces will not be sent " +
            "to Application Insights. Set it to enable local telemetry. " +
            "(This variable is auto-injected in hosted Foundry containers — do not declare it in agent.manifest.yaml.)");

    var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
        ?? throw new InvalidOperationException(
            "FOUNDRY_PROJECT_ENDPOINT environment variable is not set.");

    var model = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME")
        ?? throw new InvalidOperationException(
            "AZURE_AI_MODEL_DEPLOYMENT_NAME environment variable is not set.");

    var credential = new DefaultAzureCredential();
    var tokenScope = "https://ai.azure.com/.default";

    builder.Services.AddSingleton(new CopilotClient(new()
    {
        Mode = CopilotClientMode.Empty,
    }));

    builder.Services.AddSingleton(new ProviderConfig
    {
        Type = "azure",
        BaseUrl = $"{endpoint}/openai/v1",
        WireApi = "responses",
        // IMPROVEMENT: Use DefaultAzureCredential/GetBearerToken so it's simpler code and supports token refresh
        GetBearerToken = async _ => (await credential.GetTokenAsync(new([tokenScope]))).Token,
    });
});

/// <summary>
/// Hello World handler — forwards user input to a Foundry model via the Responses API,
/// streams the reply as SSE token events, and persists conversation history
/// in an in-memory session store keyed by <see cref="InvocationContext.SessionId"/>.
/// </summary>
/// <param name="responsesClient">Foundry Responses API client, injected via DI.</param>
/// <param name="logger">Logger injected via DI. Calls are automatically exported to Application Insights.</param>
public sealed class HelloWorldHandler(
    CopilotClient copilotClient,
    ProviderConfig providerConfig,
    ILogger<HelloWorldHandler> logger) : InvocationHandler
{
    // ── Required override ─────────────────────────────────────────────────────
    // HandleAsync is the only method you must override. It receives every
    // POST /invocations request.
    //
    // Three optional overrides exist for long-running operations (LRO):
    //   GetAsync          — handle GET /invocations/{id} status polls
    //   CancelAsync       — handle DELETE /invocations/{id} cancellation
    //   GetOpenApiAsync   — serve an OpenAPI spec at GET /invocations/docs/openapi.json
    // For a simple streaming agent like this one, none of them are needed.
    // ─────────────────────────────────────────────────────────────────────────
    public override async Task HandleAsync(
        HttpRequest request,
        HttpResponse response,
        InvocationContext context,
        CancellationToken cancellationToken)
    {
        // IMPROVEMENT: Stop requiring JSON input
        var userMessage = await new StreamReader(request.Body).ReadToEndAsync(cancellationToken);

        // InvocationContext is provided by the Invocations SDK. It resolves
        // session and invocation identity from the incoming request headers
        // so you don't have to parse them yourself.
        logger.LogInformation(
            "Processing invocation {InvocationId} (session {SessionId})",
            context.InvocationId, context.SessionId);

        // IMPROVEMENT: Detect if session exists; don't try resuming one that doesn't
        // Retrieve or create session.
        var sessionMetadata = await copilotClient.GetSessionMetadataAsync(context.SessionId, cancellationToken);

        SessionConfigBase sessionConfig = sessionMetadata is null ? new SessionConfig { SessionId = context.SessionId } : new ResumeSessionConfig();
        sessionConfig.OnPermissionRequest = PermissionHandler.ApproveAll;
        sessionConfig.Streaming = true;
        sessionConfig.Model = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME");
        sessionConfig.Provider = providerConfig;

        // IMPROVEMENT: Don't immediately resume and shut down the session on every request. Keep it alive for some period.
        await using var session = sessionMetadata is null
            ? await copilotClient.CreateSessionAsync((SessionConfig)sessionConfig, cancellationToken)
            : await copilotClient.ResumeSessionAsync(context.SessionId, (ResumeSessionConfig)sessionConfig, cancellationToken);

        response.ContentType = "text/event-stream";
        response.Headers.CacheControl = "no-cache";

        // SUGGESTION: The code below could be improved if the Foundry helper library contained methods for emitting /invocations chunks/responses instead of raw JSON
        using var subscription = session.On<AssistantMessageDeltaEvent>(async delta =>
        {
            var tokenEvent = JsonSerializer.Serialize(new { type = "token", content = delta.Data.DeltaContent });
            await response.WriteAsync($"data: {tokenEvent}\n\n", cancellationToken);
            await response.Body.FlushAsync(cancellationToken);
        });

        var reply = await session.SendAndWaitAsync(userMessage, cancellationToken: cancellationToken);

        // Send the final done event with the complete reply text.
        var doneEvent = JsonSerializer.Serialize(new
        {
            type = "done",
            invocation_id = context.InvocationId,
            session_id = context.SessionId,
            full_text = reply?.Data.Content ?? "",
        });
        await response.WriteAsync($"data: {doneEvent}\n\n", cancellationToken);
        await response.Body.FlushAsync(cancellationToken);
    }
}