.NET: AG-UI support for .NET: Support for tool calling

[javiercn]

This PR adds comprehensive support for tool calling in the AG-UI implementation for .NET, including client-side and server-side function execution, conversation management, and proper serialization across boundaries.

Key Changes

1. AGUIAgent → AGUIChatClient: Architectural Shift

Previous Architecture:

• The original design used AGUIAgent as a wrapper around an IChatClient
• This prevented using FunctionInvokingChatClient and required creating custom threads and so on.
  • IChatClient now handles the conversation on a given turn(s) and ChatClientAgent handles the run, memory, etc.

New Architecture:

• AGUIChatClient is now an IChatClient implementation that communicates with AG-UI compliant servers
• It properly extends DelegatingChatClient and wraps a FunctionInvokingChatClient
• The internal architecture now has clear layers:
  1. AGUIChatClient (outer) - manages conversation ID and ServerFunctionCallContent wrapping/unwrapping
  2. FunctionInvokingChatClient (middle) - handles client-side tool execution
  3. AGUIChatClientHandler (inner) - handles HTTP communication with AG-UI server

Why this matters:

• Composability: Leverages the existing FunctionInvokingChatClient for client-side tool execution
• Separation of concerns: HTTP communication, tool execution, and conversation management are cleanly separated
• Standards compliance: Implements IChatClient interface consistently with the rest of the ecosystem

---

2. ConversationId Handling in AG-UI

AG-UI has a fundamental constraint: it requires the full message history on every turn and uses ThreadId for conversation tracking. However, the Microsoft.Extensions.AI IChatClient interface uses ConversationId to indicate stateful conversation management where the client maintains history.

The Challenge:

• When ChatOptions.ConversationId is set, FunctionInvokingChatClient assumes the underlying client maintains history and only sends new messages
• AG-UI always needs the full history, even across multiple turns
• We need to preserve the conversation identity for the caller while preventing FunctionInvokingChatClient from treating the conversation as stateful

The Solution:
The PR implements a sophisticated "conversation ID juggling" pattern:

1. On Request (Client → Server):

   // If user provides a conversation ID, extract it and clear it from options
   if (options?.ConversationId != null)
   {
       conversationId = options.ConversationId;
       innerOptions = options.Clone();
       innerOptions.AdditionalProperties["agui_thread_id"] = conversationId;
       innerOptions.ConversationId = null;  // Hide from FunctionInvokingChatClient
   }

2. During Tool Execution:

   • For client tool calls: Store the thread ID in FunctionCallContent.AdditionalProperties["agui_thread_id"]
   • When the tool result comes back with the history, extract the thread ID from the penultimate message
   • This ensures continuity across the function invocation round-trip

3. On Response (Server → Client):

   // Extract thread ID from first update if not already set
   if (firstUpdate != null && conversationId == null)
   {
       conversationId = firstUpdate.AdditionalProperties["agui_thread_id"];
   }
   
   // Restore conversation ID for all updates before yielding to caller
   finalUpdate.ConversationId = conversationId;

Why this complex approach?

• AG-UI protocol requirement: Must send full history every time
• Microsoft.Extensions.AI design: ConversationId implies incremental history
• FunctionInvokingChatClient behavior: Only sends new messages when ConversationId is set
• Solution: Hide ConversationId from the middleware layer, use AdditionalProperties as temporary storage, restore it for the caller

This ensures:

• ✅ Callers see consistent ConversationId across all updates
• ✅ Full message history is always sent to AG-UI server
• ✅ Multi-turn conversations work correctly
• ✅ Tool calls preserve conversation context

---

3. ServerFunctionCallContent: Hiding Server-Executed Tools

The Problem:
When a server executes a tool, it returns a FunctionCallContent in the response stream. The wrapping FunctionInvokingChatClient sees this and tries to execute it again on the client side, causing:

• Duplicate execution attempts
• Errors when the client doesn't have the server's tool registered
• Confusion about where tools should execute

The Solution - ServerFunctionCallContent:

private class ServerFunctionCallContent(FunctionCallContent functionCall) : AIContent
{
    public FunctionCallContent FunctionCallContent { get; } = functionCall;
}

How it works:

1. Detection (in AGUIChatClientHandler):

   if (update.Contents[0] is FunctionCallContent fcc)
   {
       if (clientToolSet.Contains(fcc.Name))
       {
           // Client tool - let FunctionInvokingChatClient handle it
       }
       else
       {
           // Server tool - wrap it to hide from FunctionInvokingChatClient
           update.Contents[0] = new ServerFunctionCallContent(fcc);
       }
   }

2. Unwrapping (in AGUIChatClient):

   for (var i = 0; i < update.Contents.Count; i++)
   {
       if (content is ServerFunctionCallContent serverFunctionCallContent)
       {
           update.Contents[i] = serverFunctionCallContent.FunctionCallContent;
       }
   }

Why a wrapper class?

• FunctionInvokingChatClient only recognizes FunctionCallContent as something it should execute
• By wrapping server tool calls in ServerFunctionCallContent, they become "invisible" to the middleware
• The outer AGUIChatClient then unwraps them before yielding to the caller
• Result: Callers see normal FunctionCallContent, but middleware doesn't try to execute server tools

Benefits:

• ✅ Server tools execute only on the server
• ✅ Client tools execute only on the client
• ✅ Mixed server/client tool scenarios work correctly
• ✅ No duplicate execution attempts
• ✅ Clean separation of execution boundaries

---

Additional Changes in Broad Strokes

Message Type Hierarchy

• Introduced proper message type hierarchy: AGUIMessage (base) with concrete types:
  • AGUIUserMessage
  • AGUIAssistantMessage
  • AGUISystemMessage
  • AGUIDeveloperMessage
  • AGUIToolMessage
• Replaced generic AGUIMessage with role strings to strongly-typed message classes
• Improved type safety and serialization handling

Tool Call Events

• Added comprehensive tool call events:
  • ToolCallStartEvent
  • ToolCallArgsEvent
  • ToolCallEndEvent
  • ToolCallResultEvent
• Converted between ChatResponseUpdate (with FunctionCallContent) and AG-UI events
• Proper streaming of tool call arguments

JSON Serialization

• Added JsonSerializerOptions parameter to AGUIChatClient constructor
• Allows custom JSON contexts for client-specific types
• Server and client can use different serialization contexts
• Merged AGUI context with custom contexts automatically
• Proper handling of cross-boundary serialization (server types → JsonElement on client)

[Copilot]

Pull Request Overview

This pull request adds comprehensive tool calling support to the AG-UI framework, enabling both client-side and server-side function execution. The changes include:

• Refactored the AG-UI architecture to use AGUIChatClient (implementing IChatClient) instead of AGUIAgent
• Added polymorphic message types (AGUIUserMessage, AGUIAssistantMessage, AGUIToolMessage, etc.) replacing the single AGUIMessage class
• Implemented tool call event types (ToolCallStartEvent, ToolCallArgsEvent, ToolCallEndEvent, ToolCallResultEvent) for streaming tool execution
• Added extensive unit and integration tests covering tool calling scenarios
• Updated samples to demonstrate client/server tool calling capabilities

Reviewed Changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
dotnet/src/Microsoft.Agents.AI.AGUI/AGUIChatClient.cs	New IChatClient implementation with tool calling support
dotnet/src/Microsoft.Agents.AI.AGUI/Shared/ChatResponseUpdateAGUIExtensions.cs	Core conversion logic between ChatResponseUpdate and AG-UI events including tool calls
dotnet/src/Microsoft.Agents.AI.AGUI/Shared/AGUIMessage*.cs	Polymorphic message type hierarchy replacing single AGUIMessage class
dotnet/src/Microsoft.Agents.AI.AGUI/Shared/ToolCall*.cs	New event types for tool call streaming
dotnet/tests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.UnitTests/ChatResponseUpdateAGUIExtensionsTests.cs	Comprehensive unit tests for tool calling conversion
dotnet/tests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.IntegrationTests/ToolCallingTests.cs	End-to-end integration tests with Azure OpenAI
dotnet/src/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore/AGUIEndpointRouteBuilderExtensions.cs	Server-side endpoint supporting tool execution
dotnet/samples/AGUIClientServer/AGUIClient/Program.cs	Updated client sample with tool calling
dotnet/samples/AGUIClientServer/AGUIServer/Program.cs	Updated server sample with tool registration

[markwallace-microsoft]

Integration failure unrelated

  Passed OpenAIAssistant.IntegrationTests.OpenAIAssistantIRunTests.RunWithChatMessageReturnsExpectedResultAsync [6 s]
[xUnit.net 00:03:09.00]     OpenAIAssistant.IntegrationTests.OpenAIAssistantClientExtensionsTests.CreateAIAgentAsync_WithHostedCodeInterpreter_RunsCodeAsync(createMechanism: "CreateWithChatClientAgentOptionsSync") [FAIL]