Skip to content

feat(rehydrate): add max_history_bytes and max_history_items caps#336

Open
crstrn13 wants to merge 3 commits into
praxis-proxy:mainfrom
crstrn13:feat/rehydrate-limits
Open

feat(rehydrate): add max_history_bytes and max_history_items caps#336
crstrn13 wants to merge 3 commits into
praxis-proxy:mainfrom
crstrn13:feat/rehydrate-limits

Conversation

@crstrn13

@crstrn13 crstrn13 commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Closes #66.

  • Add max_history_bytes (default 2 MiB) and max_history_items (default unlimited) config fields to the openai_responses_rehydrate filter
  • Enforce limits before stored history is spliced into ResponsesState, preventing unbounded proxy memory growth
  • Reject oversized history with 413 and an actionable error message ("compact or shorten the conversation before continuing")
  • Introduce a RehydrationSource trait so build_state is generic over ResponseRecord and ConversationRecord — limits are enforced uniformly on both the previous_response_id and conversation rehydration paths
  • Restructure rehydrate into a clean dispatcher with two symmetric methods: rehydrate_from_response and rehydrate_from_conversation
  • Fix fail-open bug: serialization failure in byte-size measurement now falls back to usize::MAX (reject) instead of 0 (allow)

Config

- filter: openai_responses_rehydrate
  max_history_bytes: 2097152  # 2 MiB (default)
  max_history_items: ~        # unlimited (default)

Both fields reject zero values at config load time.

Test plan

  • 55 unit tests pass (cargo test -p praxis-ai-apis -- rehydrate)
  • Config validation: rejects zero max_history_bytes, zero max_history_items, unknown fields
  • Response path: rejects byte limit, item limit, exact boundary, streaming 413 format
  • Conversation path: rejects byte limit, item limit, within-limits passthrough
  • make lint clean (clippy + nightly fmt + filter docs)

Signed-off-by: Alexander Cristurean <acristur@redhat.com>
@crstrn13 crstrn13 force-pushed the feat/rehydrate-limits branch from 8f50651 to d610421 Compare July 13, 2026 16:55
Signed-off-by: Alexander Cristurean <acristur@redhat.com>
@crstrn13 crstrn13 changed the title feat: added enforcing on limits for rehydrate filter. feat(rehydrate): add max_history_bytes and max_history_items caps Jul 14, 2026
@crstrn13 crstrn13 marked this pull request as ready for review July 14, 2026 14:28
@crstrn13 crstrn13 requested review from a team, alexsnaps and leseb July 14, 2026 14:28

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One blocking memory-boundary concern is noted inline.

max_history_items: Option<usize>,
streaming: bool,
) -> Result<ResponsesState, FilterAction> {
let stored = source.stored_messages();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Enforce the limits before cloning the stored history. source.stored_messages() materializes the entire Vec here, while ConversationRecord/ResponseRecord have already been fully deserialized by get_conversation/get_response; enforce_history_limits therefore runs only after the potentially unbounded allocation has happened. A large accumulated conversation can still cause a substantial memory spike before returning 413, so max_history_bytes does not provide the advertised memory bound. Please enforce the item/byte limits on the borrowed stored JSON (or fetch a bounded representation) before cloning it into the request state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add defensive caps for Responses rehydrate history

2 participants