Add native Voxtral realtime streaming buffer

[rgbkrk]

Summary

• Adds a model-native Voxtral Realtime stream buffer and stream session in voice-voxtral.
• Encodes the Mistral/vLLM scheduling contract: initial streaming-pad prompt, 80 ms raw-audio token stride, lookback/lookahead audio windows, generated-token feedback, and final right padding.
• Exposes native streaming STT sessions from voice-stt for Voxtral while keeping Whisper as explicit non-native-streaming.
• Adds voice listen --stream, defaulting to Voxtral when no STT backend is selected.
• Routes voice realtime / voice realtime-tui partial STT through the native Voxtral stream session when Voxtral is loaded, with the finished stream used as the final transcript.
• Adds an ignored real-model Voxtral streaming smoke over eval/recordings/002.wav.

Verification

• cargo fmt --check
• cargo test -p voice-voxtral realtime_stream
• cargo test -p voice-voxtral realtime_session
• cargo test -p voice-voxtral
• cargo test -p voice-stt
• cargo test -p voice
• cargo check -p voice-stt
• cargo check -p voice
• cargo run -q -p voice -- listen --help
• cargo test -p voice-stt voxtral_native_stream_smoke_transcribes_eval_recording -- --ignored --nocapture

Notes

This is now true model-native Voxtral STT streaming through the local Rust stack. The current correctness baseline recomputes full text and audio prefix history each step; KV-cache and incremental audio encoder state are the next latency/perf slice.

[pullfrog]

ℹ️ Two rough edges worth a look — one is a protocol inconsistency that affects latency metrics, the other is a fragile float tolerance in the stream config validator.

Reviewed changes — adds native Voxtral Realtime STT streaming: a VoxtralRealtimeStreamBuffer that manages audio-token interleave windows, a VoxtralRealtimeStreamSession that runs single-token greedy decode per window, an SttStreamSession adapter in voice-stt, and CLI wiring for voice listen --stream and voice realtime --stt-partials with Voxtral loaded.

• VoxtralRealtimeStreamBuffer / VoxtralRealtimeStreamConfig — encodes the vLLM realtime scheduling contract: initial streaming-pad prefix, 80 ms raw-audio stride, lookback/lookahead windows, generated-token feedback queue, and right-pad on finish()
• VoxtralRealtimeStreamSession — stateful per-token decode loop; accumulates full input-token prefix on each step (KV cache deferred per doc comment); routes EOS correctly; output_tokens never includes EOS in decode_text
• SttStreamSession<'a> enum + VoxtralRealtimeSttStreamSession — resamples to 16 kHz on push_audio, wraps next_step / drain_ready / finish; stream_session() on SttModel returns Err for Whisper
• listen --stream — record_until_interrupt_with_progress with 80 ms progress interval; streams incremental audio to session; prints transcript with ANSI clear-line
• Realtime live-turn native path — when stt_partials + supports_native_streaming(), records with streaming session; falls back to transcribe_samples only if no stream result; native elapsed time accumulated across steps
• Tests — realtime_stream and realtime_session unit tests with tiny synthetic config; voxtral_native_stream_smoke_transcribes_eval_recording integration test (ignored, requires model weights)

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using us.anthropic.claude-sonnet-4-6 ｜ 𝕏}

[pullfrog]

"elapsed_ms": 0 is hardcoded, whereas the non-native Whisper partial path (see maybe_emit_realtime_partial_transcription) computes and emits the real per-step inference wall time in the same conversation.item.input_audio_transcription.partial event shape. Any consumer recording latency metrics will silently get 0 for native streaming turns and a real value for Whisper turns — same field, two incompatible semantics.

The native session doesn't have a single inference-start here, but the per-step cost is already tracked in native_stream_elapsed_ms. Either propagate a per-step inference duration into NativeStreamPartialContext or emit null to make the absence explicit rather than lying with 0.

[pullfrog]

Frame-rate equality guard uses f64::EPSILON (~2.2e-16) as tolerance. 12.5 Hz is representable exactly in IEEE 754 so this passes in practice, but the choice of machine epsilon for a physical quantity comparison is fragile — any future frame rate that isn't exactly representable (e.g. 1000.0/75.0 ≈ 13.333...) would need either exact representation or a looser guard.

Prefer a physically-meaningful epsilon like 1e-6, which covers legitimate float representations of frame rates while still catching mismatched configs (12.5 vs 25.0).