Responses stateful streaming: buffering, fail-closed, and orchestration

[leseb]

Parent

Part of #101

Scope

Stateful streaming business logic on top of the SSE parser (#501):

• Accumulate response.function_call_arguments.delta by output item/index/call
• Require a matching response.function_call_arguments.done before executing a tool
• Enforce terminal event handling (response.completed, response.failed, response.incomplete)
• Fail closed on truncated streams, incomplete tool-call arguments, or missing terminal events
• Default policy: buffered_final_stream — consume upstream SSE internally, buffer tool-call events, return valid final output to client
• Synthesize downstream output:
  • Valid Responses JSON when returning a non-streamed final response
  • Valid Responses SSE when returning a streamed final response
• Preserve final response metadata (response id, model, status, output text)
• Future-compatible progress_events mode (do not forward tool-call events to client unless explicitly configured)

Acceptance criteria

• Interleaved function-call argument deltas are buffered by the correct output item/call
• Incomplete function calls fail closed before any tool execution
• Missing terminal events fail closed
• Final-text-only streamed model response produces a valid downstream response
• Tool-call streamed model response executes the tool only after complete arguments are available
• Synthesized downstream JSON/SSE is valid Responses-shaped output
• Stateless SSE pass-through remains unchanged

Dependencies

• Depends on Responses SSE parser: chunk reassembly and event parsing #7 (SSE parser)

Reference

• Proposal: Responses API Filters — Filter 6
• Part of Epic: Responses API and Agentic Loop Orchestration #93

[leseb]

Partial coverage from stream_events (#270)

The openai_stream_events filter (PR #270) covers the accumulation and detection half of this issue:

Landed:

• Accumulate function_call_arguments.delta by output item/index/call
• Enforce terminal event handling (response.completed, response.failed, response.incomplete)
• Detect truncated streams and missing terminal events (sets responses.stream_incomplete metadata)
• Preserve final response metadata in ResponsesState

Remaining scope (depends on tool_dispatch #26):

• Require matching arguments.done before executing a tool (fail-closed enforcement)
• buffered_final_stream policy — consume upstream SSE internally, buffer tool-call events, return valid final output to client
• Synthesize downstream JSON/SSE responses
• progress_events mode
• Active fail-closed behavior (reject/error, not just metadata flagging)

The remaining items are ReadWrite/mutating behavior that belongs in the orchestration layer, not the read-only observer.

[leseb]

Will come after #23 and #26