docs: add v0.8.5 changelog — content-based routing, A/B testing, per-…#3
Conversation
…key rate limiting, budget plugin
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
Adds the v0.8.5 release notes to the documentation changelog, highlighting new routing strategies and new traffic-control plugins/config options.
Changes:
- Updated the changelog page meta description to include newly released features.
- Added a new v0.8.5 section covering content-based routing, A/B testing, per-key/per-user rate limiting, and a budget plugin.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ### Per-key and per-user rate limiting | ||
|
|
||
| - Extended `rate-limit` plugin with `key_rpm` (requests per minute per API key) and `user_rpm` (requests per minute per user ID) | ||
| - Rate checks execute in order: global → per-key → per-user; request rejected at first exceeded limiter with a distinct reason string |
There was a problem hiding this comment.
This entry adds key_rpm/user_rpm options for the rate-limit plugin, but the Rate limiting guide currently documents only requests_per_second/burst. To avoid conflicting configuration references, update the rate limiting docs to include the new per-key/per-user limiters and their precedence/"reason" strings.
| - Rate checks execute in order: global → per-key → per-user; request rejected at first exceeded limiter with a distinct reason string | |
| - Rate checks execute in order: global (`requests_per_second`/`burst`) → per-key (`key_rpm`) → per-user (`user_rpm`); the first exceeded limiter rejects the request and sets `reason` to `rate_limit_global`, `rate_limit_key`, or `rate_limit_user` respectively |
| ### Per-key budget controls plugin | ||
|
|
||
| - New `budget` plugin tracks cumulative USD spend per API key in an in-memory store | ||
| - Register at `before_request` to reject over-limit keys and `after_request` to record token costs | ||
| - Two instances sharing the same `store_id` share accumulated spend data | ||
| - `spend_limit_usd: 0` (or unset) means unlimited — spend is tracked without rejection | ||
| - Spend data is in-memory; does not survive process restarts |
There was a problem hiding this comment.
Changelog announces a new budget plugin, but the Plugins guide frontmatter and content still describe "11 built-in plugins" and does not include a budget plugin section. Please update the plugins documentation (and the count in its description) so the new plugin is discoverable.
|
|
||
| ### Content-based routing strategy | ||
|
|
||
| - New `strategy.mode: content-based` selects a provider target based on user-role prompt content |
There was a problem hiding this comment.
Minor wording: "user-role prompt content" reads like a typo/awkward hyphenation. Consider rephrasing to "user role" or "user/role" prompt content for clarity.
| - New `strategy.mode: content-based` selects a provider target based on user-role prompt content | |
| - New `strategy.mode: content-based` selects a provider target based on user role prompt content |
| - Regex patterns compiled at startup for zero-cost hot-path matching; invalid patterns surface as a startup error | ||
|
|
||
| ### A/B testing strategy | ||
|
|
||
| - New `strategy.mode: ab-test` splits traffic across two or more named variants using weighted random sampling | ||
| - Each variant carries a `label` (e.g. `"control"`, `"challenger"`) emitted as the `ab_variant` structured log field on every routed request | ||
| - Zero-weight variants participate with weight 1 (equal distribution) |
There was a problem hiding this comment.
Changelog introduces new routing strategies (content-based, ab-test), but the Routing policies guide still states there are only six strategies and does not mention these modes. Please update the routing strategy documentation (or link to the relevant page/section) so readers can find configuration details for the new modes.
| - Regex patterns compiled at startup for zero-cost hot-path matching; invalid patterns surface as a startup error | |
| ### A/B testing strategy | |
| - New `strategy.mode: ab-test` splits traffic across two or more named variants using weighted random sampling | |
| - Each variant carries a `label` (e.g. `"control"`, `"challenger"`) emitted as the `ab_variant` structured log field on every routed request | |
| - Zero-weight variants participate with weight 1 (equal distribution) | |
| - Regex patterns compiled at startup for zero-cost hot-path matching; invalid patterns surface as a startup error | |
| - See the [Routing policies guide](./routing-policies) for full configuration options for `content-based` strategies. | |
| ### A/B testing strategy | |
| - New `strategy.mode: ab-test` splits traffic across two or more named variants using weighted random sampling | |
| - Each variant carries a `label` (e.g. `"control"`, `"challenger"`) emitted as the `ab_variant` structured log field on every routed request | |
| - Zero-weight variants participate with weight 1 (equal distribution) | |
| - See the [Routing policies guide](./routing-policies) for full configuration options for `ab-test` strategies and the complete list of supported modes. |
…per-key/user rate limiting, budget plugin
…key rate limiting, budget plugin
What does this PR change?
Type of change
Checklist
pnpm buildpasses locally (no broken links)pnpm typecheckpassestitle,description, andkeywordson any new/modified pagesidebars.tsdevelopmentbranch (notmain)Related issue