[BUG] CompleteChatAsync fails to parse GPT-4o-mini responses at OpenAI.Chat.ChatCompletion.FromResponse(PipelineResponse response)

[MaireadMaloneyProfessional]

Service

OpenAI

Describe the bug

When calling ChatClient.CompleteChatAsync with a GPT-4o-mini deployment, we randomly see calls fail with a parsing error.

StackTrace:

at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan1 bytes) at System.Text.Json.Utf8JsonReader.ConsumeNextToken(Byte marker) at System.Text.Json.Utf8JsonReader.ReadSingleSegment() at System.Text.Json.JsonDocument.Parse(ReadOnlySpan1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack)
at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter) at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory1 utf8Json, JsonDocumentOptions options)
at OpenAI.Chat.ChatCompletion.FromResponse(PipelineResponse response)
at OpenAI.Chat.ChatClient.CompleteChatAsync(IEnumerable`1 messages, ChatCompletionOptions options, CancellationToken cancellationToken)

Exception: '{' is invalid after a single JSON value. Expected end of data. LineNumber: 1 | BytePositionInLine: 0.

This has started happening only after moving from gpt-35-turbo-16k to gpt-4o-min.

It is happening 0.5% of all our calls to OpenAI, across different prompts.

Steps to reproduce

1. Use a gpt-4o-mini to create an instance of the ChatClient in the OpenAI .NET SDK v.2.0.0
2. Call ChatClient.CompleteChatAsync
3. Repeat until you encounter exception "'{' is invalid after a single JSON value. Expected end of data. LineNumber: 1 | BytePositionInLine: 0."

Code snippets

OS

winOS

.NET version

8

Library version

2.0.0