Skip to content

fix: add content-based session hash fallback for OpenAI compat clients#1459

Open
octo-patch wants to merge 1 commit intoWei-Shaw:mainfrom
octo-patch:fix/issue-1421-openai-content-based-session-hash
Open

fix: add content-based session hash fallback for OpenAI compat clients#1459
octo-patch wants to merge 1 commit intoWei-Shaw:mainfrom
octo-patch:fix/issue-1421-openai-content-based-session-hash

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #1421

Problem

OpenAIGatewayService.GenerateSessionHash only checks three explicit session signals (headers session_id/conversation_id, body field prompt_cache_key). When none are present — which is the case for standard OpenAI-compatible clients such as Claude Code IDE extensions, custom scripts, and most non-Codex CLI tools — the function returns an empty string.

An empty session hash means SelectAccountWithScheduler picks a random upstream account on every request, preventing prompt caching entirely. Successive turns of the same conversation land on different accounts, so the upstream cache is never reused.

Solution

Add a content-based fallback (priority 4) that builds a stable seed from:

  • Authorization header (SHA-256 hashed before use, so different API keys route independently)
  • model field
  • All system messages
  • First user message only

Using only the stable request prefix — rather than the full growing message list — ensures the hash remains constant across conversation turns. This gives sticky-session routing for the lifetime of a conversation without breaking when new messages are appended.

Clients that already provide explicit session signals (priorities 1-3) are completely unaffected.

Testing

  • Added TestOpenAIGatewayService_GenerateSessionHash_ContentBasedFallback covering:
    • Content-based hash is non-empty when Authorization + content are present
    • Same content + same auth → identical hash (retry stability)
    • Different auth header → different hash (user isolation)
    • Growing conversation (more messages) → same hash (stable prefix)
    • Empty body + no auth → empty hash (no change to existing no-signal behaviour)
    • Explicit session_id header still takes priority over content-based fallback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[fix] OpenAIGatewayService.GenerateSessionHash` lacks content-based fallback, causing cache misses for non-Codex clients like Claude Code

1 participant