Race Condition: ChatClient.CompleteChatAsync Mutates Shared ChatCompletionOptions, Causing Incorrect Request Payloads in Parallel Calls

[chi-qin]

Service

OpenAI

Describe the bug

When using ChatClient.CompleteChatAsync (or CompleteChatStreamingAsync) within a parallel loop (e.g., Parallel.ForEachAsync), if the same ChatCompletionOptions instance is passed to multiple concurrent calls, a race condition occurs. The internal method CreateChatCompletionOptions (or a similar method responsible for preparing the request) modifies the Messages property (and potentially others) of the passed-in ChatCompletionOptions object by reference.

This leads to a scenario where the Messages (and thus the content of the API request) for one parallel operation can be overwritten by another before the request is fully constructed and sent. As a result, multiple distinct input messages can end up being processed by the OpenAI API as if they were the same, often the message from the last thread to modify the shared options object before requests are dispatched.

Steps to reproduce

Code snippets

using Azure.AI.OpenAI;
using Azure.Identity;
using OpenAI.Chat;

ChatCompletionOptions _options = new()
{
    Temperature = 0,
};

AzureOpenAIClient _azureClient = new(new Uri("https://learn-shared-aoai-test-eus2.openai.azure.com/"), new DefaultAzureCredential(), new AzureOpenAIClientOptions());

ChatClient _chatClient = _azureClient.GetChatClient("gpt-4.1-2025-04-14");

IReadOnlyList<string> _messages =
[
    "2 + 4 = ?",
    "3 + 5 = ?",
    "4 + 6 = ?"
];

await Parallel.ForEachAsync(_messages, async (message, cancellationToken) =>
{
    List<ChatMessage> messages =
    [
        new UserChatMessage(message),
    ];

    ChatCompletion completion = await _chatClient.CompleteChatAsync(messages, _options, cancellationToken);

    Console.WriteLine($"Message: {message}, Response: {completion.Content[0].Text}");
});

OS

windows

.NET version

9.0.204

Library version

2.1.0

[jsquire]

Hi @chi-qin. Thanks for reaching out and we regret that you're experiencing difficulties. This is indeed a bug, as we should not be mutating parameters that are not explicitly marked as ref or out - or we explicitly document the mutation. In this case, the method is mutating internal state directly rather than working against a private clone.

We'll get this fixed up. In the meantime, mitigation would be to ensure that you're using a new instance of the options with each chatClient conversation.