fix(inference): support request-level provider api key

[Vaibhav701161]

Summary

Adds request-level BYOK support for provider API keys through x-bf-provider-api-key.

The goal is to let a request use a customer-provided provider key without storing it in Bifrost and without bypassing the normal provider/model/key selection flow. Bifrost still selects an eligible configured key first, then only overrides the selected key value for that single request.

Changes

• Added x-bf-provider-api-key as a request context key.
• Read the header from HTTP requests and store it only in the request context.
• Override only the selected key value after normal key selection is complete.
• Preserved selected key metadata like ID, name, model filtering, and key eligibility.
• Added x-bf-provider-api-key to security deny lists so it cannot be forwarded through extra headers.
• Added it to security header filtering paths.

Main trade-off: usage/key attribution still points to the selected configured key, since only the secret value is replaced for the request. This keeps the change small and avoids introducing customer key storage.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Build:

cd transports/bifrost-http
CGO_ENABLED=1 go build -ldflags="-w -s -X main.Version=v0.0.0-dev" -tags "sqlite_static" -o ../../tmp/bifrost-http .

Runtime validation was done with Bifrost running on 127.0.0.1:9090 and a local capture provider on 127.0.0.1:19091.

Without BYOK header:

curl -sS -X POST http://127.0.0.1:9090/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"hello"}]}' >/dev/null

cat /tmp/bifrost_byok_capture.jsonl

Expected: downstream provider receives the configured key.

With BYOK header:

curl -sS -X POST http://127.0.0.1:9090/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'x-bf-provider-api-key: CUSTOMER_KEY_SHOULD_BE_USED' \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"hello"}]}' >/dev/null

cat /tmp/bifrost_byok_capture.jsonl

Expected: downstream provider receives Bearer CUSTOMER_KEY_SHOULD_BE_USED.

Then removing the header again switches back to the configured key.

Also verified:

• BYOK does not allow unconfigured providers.
• BYOK does not allow unsupported models.
• Customer key was not present in Bifrost logs.
• Customer key was not present in response bodies.

Screenshots/Recordings

Added terminal screenshot showing:

• request without x-bf-provider-api-key uses the configured vendor key
• request with x-bf-provider-api-key uses the request-scoped customer key
• request without the header again goes back to the configured vendor key
• unsupported provider is still blocked

Breaking changes

• Yes
• No

Related issues

closes BF-557

Security considerations

This touches provider API key handling, so I kept the scope request-only.

• Raw customer provider key is not stored.
• Raw customer provider key is not written to provider config.
• Normal provider/model/key eligibility still runs before the override.
• x-bf-provider-api-key is blocked from extra-header forwarding paths.
• Runtime check confirmed the dummy customer key did not appear in Bifrost logs or response bodies.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6559bf80-5d6a-4ce0-aa54-be2ff33255d1

📥 Commits

Reviewing files that changed from the base of the PR and between fdcacc5 and 7563ade.

📒 Files selected for processing (5)

• core/bifrost.go
• core/schemas/bifrost.go
• transports/bifrost-http/handlers/config.go
• transports/bifrost-http/handlers/wsresponses.go
• transports/bifrost-http/lib/ctx.go

✅ Files skipped from review due to trivial changes (1)

• core/schemas/bifrost.go

🚧 Files skipped from review as they are similar to previous changes (2)

• transports/bifrost-http/handlers/config.go
• core/bifrost.go

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added request-scoped provider API key overrides: an override replaces only the key secret while preserving the selected key's identity and eligibility, and is applied per-request.

• Bug Fixes / Behavior

  • When an override is active, key rotation is avoided on retry after rate limits and override-aware logic ensures retries use the overridden secret without marking keys as used.

• Security Enhancements

  • Broadened blocking of API-key related headers across HTTP and WebSocket flows to prevent unintended header forwarding.

Walkthrough

This PR introduces request-scoped provider API key override via the x-bf-provider-api-key header: the header is extracted into context, a helper replaces only the selected key's secret with the override, retries skip rotation when overridden, and header propagation paths are denylisted.

Changes

Provider API Key Override

Layer / File(s)	Summary
Context key and header parsing core/schemas/bifrost.go, transports/bifrost-http/lib/ctx.go	Context key constant BifrostContextKeyProviderAPIKey is defined; HTTP header parsing extracts x-bf-provider-api-key, stores the trimmed value in context, and extends the security denylist to block key-related headers from general processing.
Override application logic core/bifrost.go	Adds applyProviderAPIKeyOverride to replace only Key.Value with the override wrapped as EnvVar. SelectKeyForProviderRequestType and executeRequestWithRetries apply this override; retries skip rotation and used-key marking when the override is active.
Security header enforcement transports/bifrost-http/handlers/config.go, transports/bifrost-http/handlers/wsresponses.go	x-bf-api-key-id and x-bf-provider-api-key are added to the securityHeaders blocklist and WS extra-header denylist, preventing unauthorized exposure or forwarding.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant HTTP as ConvertToBifrostContext
  participant Selector as SelectKeyForProviderRequestType
  participant Override as applyProviderAPIKeyOverride
  participant Runner as executeRequestWithRetries
  Client->>HTTP: sends `x-bf-provider-api-key` header
  HTTP->>Selector: context (with override)
  Selector->>Override: selectedKey + context
  Override-->>Selector: key with secret replaced
  Selector->>Runner: selectedKey (override applied)
  loop retry attempts
    Runner->>Override: selectedKey + context
    Override-->>Runner: per-attempt currentKey (override applied)
    Runner->>Runner: on rate-limit, skip rotation if override present
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• akshaydeo

Poem

🐇 I nibble headers, soft and sly,
A secret tucked where carrots lie.
I swap the filling, keep the shell,
Retry the hop, and guard it well.
Hop, hide, and watch the pathways pry.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'fix(inference): support request-level provider api key' directly describes the main feature added—request-scoped BYOK via x-bf-provider-api-key header.
Description check	✅ Passed	The description covers all major template sections: Summary, Changes, Type of change, Affected areas, How to test with build and curl commands, Screenshots, Breaking changes, Related issues, Security considerations, and a mostly-complete Checklist.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/request-level-byok

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/bifrost.go`:
- Around line 5317-5319: When applyProviderAPIKeyOverride(...) returns an
override (i.e., a request-scoped BYOK is active) set a request-scoped flag in
the context (e.g., ctx.SetValue(schemas.BifrostContextKeyRequestScopedBYOK,
true)) right after currentKey is set, and ensure the rate-limit/retry logic that
does configured-key rotation (the code path around the 429 handling / attempt
rotation logic) checks that flag and, if true, skips rotating configured keys
and uses the actual upstream credential from currentKey (and/or ctx value
BifrostContextKeySelectedKeyID) for retries; in short, mark when a header
override is active in applyProviderAPIKeyOverride and update the rotation/retry
code to no-op configured-key rotation when that flag is present.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8949c6a9-ff76-49b3-90e0-0f52c215ca33

📥 Commits

Reviewing files that changed from the base of the PR and between 71d8375 and fdcacc5.

📒 Files selected for processing (5)

• core/bifrost.go
• core/schemas/bifrost.go
• transports/bifrost-http/handlers/config.go
• transports/bifrost-http/handlers/wsresponses.go
• transports/bifrost-http/lib/ctx.go

[greptile-apps]

Confidence Score: 5/5

The change is safe to merge; the override is strictly request-scoped, no customer key touches storage or logs, and the previously reported WebSocket gap is now closed.

All three transport paths (HTTP, WebSocket createBifrostContextFromAuth, and the retry loop) consistently read and apply the BYOK value. The key struct copy is shallow but safe because only the Value (non-pointer EnvVar) field is replaced. Rate-limit rotation is correctly suppressed for BYOK requests. Security deny lists are updated in all required locations. No bugs or security issues were found.

No files require special attention.

Important Files Changed

Filename	Overview
core/bifrost.go	Adds applyProviderAPIKeyOverride helper and integrates BYOK into both SelectKeyForProviderRequestType and executeRequestWithRetries; rate-limit rotation and used-key tracking are correctly suppressed when BYOK is active.
core/schemas/bifrost.go	Adds BifrostContextKeyProviderAPIKey constant; no issues.
transports/bifrost-http/handlers/config.go	Adds x-bf-api-key-id (previously missing) and x-bf-provider-api-key to securityHeaders deny list; no issues.
transports/bifrost-http/handlers/wsresponses.go	Adds x-bf-provider-api-key case in createBifrostContextFromAuth (fixing the previously flagged WebSocket gap) and blocks the header in isSecurityDeniedExtraHeader; no issues.
transports/bifrost-http/lib/ctx.go	Adds x-bf-provider-api-key to the extra-header deny map and reads the header into BifrostContextKeyProviderAPIKey; return true correctly consumes the header before any forwarding logic runs.

_{Reviews (2): Last reviewed commit: "fix(inference): support request-level pr..." | Re-trigger Greptile}

[akshaydeo]

we are doing this 2 times. once here and once in apply overrrides - can we change this pease

The merge-base changed after approval.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
8 out of 15 committers have signed the CLA.

✅ impoiler
✅ Madhuvod
✅ Javtor
✅ BearTS
✅ binbandit
✅ Vaibhav701161
✅ d3lm
✅ kevinpdev
❌ akshaydeo
❌ Pratham-Mishra04
❌ TejasGhatte
❌ roroghost17
❌ danpiths
❌ etnperlong
❌ SahilChoudhary22
_{You have signed the CLA already but the status is still pending? Let us recheck it.}