Realtime Client Proposal - #7285

Merged

tarekgh merged 95 commits into

dotnet:mainfrom

tarekgh:RealtimeClientProposal

Mar 11, 2026

Commits on Feb 11, 2026

Realtime Client Proposal
Tarek Mahmoud Sayed
committed
4c04354
Replace [Experimental("MEAI001")] with DiagnosticIds.Experiments.AIRealTime
Tarek Mahmoud Sayed
committed
d6d6671
Apply suggestions from code review

tarekgh
and
Copilot
authored
07947cc
Remove AsIRealtimeClient extension method
Tarek Mahmoud Sayed
committed
f2fe352
Address PR review comments
Tarek Mahmoud Sayed
committed
4f9f05d
Fix multiple issues found during code review
Tarek Mahmoud Sayed
committed
ff14b19

Commits on Feb 12, 2026

Remove duplicate internal FunctionInvocationStatus enum
Tarek Mahmoud Sayed
committed
770fa28
Remove stale CreateToolsMap calls before the loop
Tarek Mahmoud Sayed
committed
3cbb1d2
Remove FunctionInvocationResultInternal; use public FunctionInvocationResult directly
Tarek Mahmoud Sayed
committed
5511b8d
Add IAsyncDisposable to IRealtimeSession; store and await receive task in OpenAIRealtimeSession
Tarek Mahmoud Sayed
committed
fbdc7cb
Revert API baseline JSON changes
Tarek Mahmoud Sayed
committed
f33e790
Clarify audio/text token counts are subsets of Input/OutputTokenCount
Tarek Mahmoud Sayed
committed
7dad147

Commits on Feb 13, 2026

Replace ToolChoiceMode enum with ChatToolMode for cross-modality consistency
Tarek Mahmoud Sayed
committed
d66e6fe
Unify TranscriptionOptions across Realtime and SpeechToText
Tarek Mahmoud Sayed
committed
fc5341e
Use HasTopLevelMediaType instead of StartsWith for audio check
Tarek Mahmoud Sayed
committed
77449ba

Commits on Feb 14, 2026

Add API compat suppressions for SpeechToTextOptions breaking changes
Tarek Mahmoud Sayed
committed
f7fe2ef
Replace string Eagerness with SemanticEagerness struct (ChatRole pattern)
Tarek Mahmoud Sayed
committed
a6b4ccd

Commits on Feb 22, 2026

Remove InternalsVisibleToTest from Microsoft.Extensions.AI.OpenAI

Tarek Mahmoud Sayed
and
Copilot
committed
03d7648
Add RawRepresentationFactory to RealtimeSessionOptions
Tarek Mahmoud Sayed
committed
5845991
Remove OpenAI-specific tracing properties from RealtimeSessionOptions
Tarek Mahmoud Sayed
committed
63531b5
Remove AIFunction/HostedMcpServerTool properties, use ChatToolMode
Tarek Mahmoud Sayed
committed
f333eaa
Improve XML docs for Usage on response and transcription messages
Tarek Mahmoud Sayed
committed
4a6947e
Remove leftover temp files
Tarek Mahmoud Sayed
committed
29bbdca
Improve ConversationId XML docs on RealtimeServerResponseCreatedMessage
Tarek Mahmoud Sayed
committed
ab0e74d
Split Text/Audio properties on RealtimeServerOutputTextAudioMessage
Tarek Mahmoud Sayed
committed
39a042d
Replace RealtimeServerMessageType enum with readonly struct
Tarek Mahmoud Sayed
committed
46ae23d
Move Parameter into ErrorContent.Details on error messages
Tarek Mahmoud Sayed
committed
156b376
Add RawRepresentation property to RealtimeContentItem
Tarek Mahmoud Sayed
committed
f59afd6
Rename Metadata to AdditionalProperties for consistency
Tarek Mahmoud Sayed
committed
e8393c1
Improve MaxOutputTokens XML docs to clarify modality scope
Tarek Mahmoud Sayed
committed
6aae7d3
Improve XML docs on RealtimeClientResponseCreateMessage properties
Tarek Mahmoud Sayed
committed
c7f041b
Improve RealtimeClientResponseCreateMessage class-level XML doc
Tarek Mahmoud Sayed
committed
cde880a
Rename EventId to MessageId for terminology consistency
Tarek Mahmoud Sayed
committed
d2516c2
Remove blank line between using directives in audio buffer messages
Tarek Mahmoud Sayed
committed
1d15400

Commits on Feb 23, 2026

Rename RealtimeAudioFormat.Type to MediaType for consistency
Tarek Mahmoud Sayed
committed
0ca0a17
Refactor IRealtimeSession: rename InjectClientMessageAsync to SendClientMessageAsync and remove updates parameter from GetStreamingResponseAsync

Tarek Mahmoud Sayed
and
Copilot
committed
03014ce
Make RealtimeSessionOptions and related types immutable with init-only properties and IReadOnlyList
Tarek Mahmoud Sayed
committed
02fd006
Fix XML doc param order, add null validation to constructors
Tarek Mahmoud Sayed
committed
aa17989

Commits on Mar 3, 2026

Merge branch 'main' into RealtimeClientProposal
tarekgh
authored
e24108b
OpenAI Realtime Provider using OpenAI SDK
Tarek Mahmoud Sayed
committed
c7dbcf8
Fix typo: add missing space in comment ('IDsince' -> 'ID since')
Tarek Mahmoud Sayed
committed
756fd75
Make CreateSessionAsync return non-nullable IRealtimeSession
Tarek Mahmoud Sayed
committed
5c61540

Commits on Mar 4, 2026

Remove SemanticEagerness from Abstractions; move eagerness to AdditionalProperties
Tarek Mahmoud Sayed
committed
82e0b4c
Include CompatibilitySuppressions for SpeechToTextOptions API changes
Tarek Mahmoud Sayed
committed
dfb684a
Remove MCP-specific types from abstractions; add provider contract docs
Tarek Mahmoud Sayed
committed
faa6cfb
Remove trailing blank line in RealtimeServerErrorMessage
Tarek Mahmoud Sayed
committed
1be4dba
Clarify ErrorMessageId XML doc to distinguish from base MessageId
Tarek Mahmoud Sayed
committed
381dc44
Rename ErrorMessageId to OriginatingMessageId for clarity
Tarek Mahmoud Sayed
committed
e186ab8
Change ExcludeFromConversation from bool to bool? for provider-default consistency
Tarek Mahmoud Sayed
committed
f752d83
Remove blank lines between Experimental attribute and class declaration
Tarek Mahmoud Sayed
committed
8a6754d
Add null validation to Content property setter
Tarek Mahmoud Sayed
committed
7c5eeb2
Make RealtimeAudioFormat.SampleRate non-nullable and remove misleading doc
Tarek Mahmoud Sayed
committed
d8e8e5f
Rename IRealtimeSession to IRealtimeClientSession and SendClientMessageAsync to SendAsync
Tarek Mahmoud Sayed
committed
621d49d
Fix null DefaultConversationConfiguration when ExcludeFromConversation is unset
Tarek Mahmoud Sayed
committed
6d50fec

Commits on Mar 5, 2026

Remove IDisposable from IRealtimeClientSession, keep only IAsyncDisposable
Tarek Mahmoud Sayed
committed
7269c15
Remove ConversationId from RealtimeServerResponseCreatedMessage
Tarek Mahmoud Sayed
committed
8843a12
Rename ResponseCreateMessage and ConversationItemCreateMessage to CreateResponseMessage and CreateConversationItemMessage
Tarek Mahmoud Sayed
committed
f372c42
Rename RealtimeContentItem to RealtimeConversationItem
Tarek Mahmoud Sayed
committed
0c8fe65
Remove VoiceSpeed from RealtimeSessionOptions and use RawRepresentationFactory seed pattern
Tarek Mahmoud Sayed
committed
df8d2c0
Remove UpdateAsync from IRealtimeClientSession, use message-based approach
Tarek Mahmoud Sayed
committed
0e9c3b8
Rename realtime session classes to include Client in name
Tarek Mahmoud Sayed
committed
5ffae0c
Simplify DisposeAsyncCore to return ValueTask directly
Tarek Mahmoud Sayed
committed
ddd058b
Convert RealtimeSessionKind from enum to extensible string struct
Tarek Mahmoud Sayed
committed
0161bf7
Remove NoiseReductionOptions from abstraction
Tarek Mahmoud Sayed
committed
d16fa7c
Remove VoiceActivityDetection hierarchy from abstraction
Tarek Mahmoud Sayed
committed
a2dd558

Commits on Mar 6, 2026

Remove AnonymousDelegatingRealtimeClientSession and anonymous Use overload
Tarek Mahmoud Sayed
committed
0f160d2
Fix JSON injection in MessageId by using JsonSerializer.Serialize
Tarek Mahmoud Sayed
committed
2f4a0b2
Remove meaningless duration metric from SendAsync in OTel session
Tarek Mahmoud Sayed
committed
50a773c
Refactor builder to operate on IRealtimeClient instead of IRealtimeClientSession
Tarek Mahmoud Sayed
committed
5932110
Rename message classes: move RealtimeClient/RealtimeServer before Message
Tarek Mahmoud Sayed
committed
39ad4b0
Merge branch 'upstream-main' into RealtimeClientProposal
Tarek Mahmoud Sayed
committed
1013c1d
Fix cancellation tests to also accept WebSocketException on net462
Tarek Mahmoud Sayed
committed
b8d5741

Commits on Mar 7, 2026

Remove PreviousId from CreateConversationItemRealtimeClientMessage as OpenAI-specific
Tarek Mahmoud Sayed
committed
51213de
Document that ResponseId may be null and response lifecycle events may be synthesized
Tarek Mahmoud Sayed
committed
056f19b
Document that CreateResponseRealtimeClientMessage may be a no-op for VAD-driven providers
Tarek Mahmoud Sayed
committed
2075355
Add RealtimeResponseStatus constants and document interruption/barge-in via Status property
Tarek Mahmoud Sayed
committed
712eb7e
Remove session parameter from RealtimeSessionOptions.RawRepresentationFactory
Tarek Mahmoud Sayed
committed
046a71b

Commits on Mar 9, 2026

Fix AOT compatibility errors in OpenAIRealtimeClientSession by using source-generated JSON serialization
Tarek Mahmoud Sayed
committed
6d4edde
Merge remote-tracking branch 'upstream/main' into RealtimeClientProposal
Tarek Mahmoud Sayed
committed
cf7c984
Fix SpeechToTextOptionsTests for unified TranscriptionOptions after upstream merge
Tarek Mahmoud Sayed
committed
b8a7c4a

Commits on Mar 10, 2026

Revert SpeechToTextOptions changes per review feedback
Tarek Mahmoud Sayed
committed
6ff1f88
Use JsonInclude/Core pattern for experimental UsageDetails properties
Tarek Mahmoud Sayed
committed
d074823
Fix SendAsync to propagate exceptions instead of swallowing them
Tarek Mahmoud Sayed
committed
ded7128
Replace toolMap dictionary with FindTool delegate pattern
Tarek Mahmoud Sayed
committed
2c7a1ce
Use IEnumerable<AITool> in FindTool/HasAnyTools to avoid unsafe IList cast
Tarek Mahmoud Sayed
committed
5c6a8c4
Document that function invocation blocks message processing loop
Tarek Mahmoud Sayed
committed
36dbe01
Add distinct ConversationItem message types to prevent double invocation
Tarek Mahmoud Sayed
committed
0a288dc
Remove custom ToolChoice telemetry attribute
Tarek Mahmoud Sayed
committed
75734cd

Commits on Mar 11, 2026

Add null validation to Item property setter
Tarek Mahmoud Sayed
committed
f6039ab
Add <exception> XML doc tags to realtime types
Tarek Mahmoud Sayed
committed
1f217b1
Use OrdinalIgnoreCase comparer for MCP headers dictionary
Tarek Mahmoud Sayed
committed
df1c338
Change RealtimeResponseStatus from const fields to static properties
Tarek Mahmoud Sayed
committed
73443ba
Add GetRequiredService extension methods for realtime types
Tarek Mahmoud Sayed
committed
a18403b
Remove unused System.Threading.Channels package reference
Tarek Mahmoud Sayed
committed
76fdedd
Update OTel semantic conventions version reference from v1.39 to v1.40
Tarek Mahmoud Sayed
committed
0dc6bca