feat: Switch to OpenAI Realtime API for voice calls#3
Open
Bartok9 wants to merge 1 commit into
Open
Conversation
Replaces the previous turn-based speak/gather flow with real-time bidirectional audio streaming via OpenAI's Realtime API. Architecture: - Custom server.js with WebSocket support for /media-stream - Telnyx media streaming (bidirectional RTP) → OpenAI Realtime - Server-side VAD for natural turn detection - g711_ulaw codec for telephony compatibility Changes: - Add server.js with WebSocket + Next.js - Update voice webhook to use streaming answer - Add OpenAIRealtimeBridge helper class - Update package.json with ws dependency - Add .env.example with required config Closes: Hume EVI reliability issues
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the turn-based speak/gather flow with real-time bidirectional audio streaming via OpenAI's Realtime API.
Why
The previous Hume EVI integration was experiencing reliability issues (~7 second error). OpenAI Realtime provides:
Architecture
Changes
server.jswith WebSocket + Next.js supportOpenAIRealtimeBridgehelper classpackage.jsonwithwsdependency.env.examplewith required configTesting
npm run dev/media-streamRequired Environment Variables
TELNYX_API_KEYOPENAI_API_KEYOPENAI_VOICE(optional, defaults to 'alloy')