feat(rehydrate): add max_history_bytes and max_history_items caps#336
feat(rehydrate): add max_history_bytes and max_history_items caps#336crstrn13 wants to merge 3 commits into
Conversation
Signed-off-by: Alexander Cristurean <acristur@redhat.com>
8f50651 to
d610421
Compare
Signed-off-by: Alexander Cristurean <acristur@redhat.com>
franciscojavierarceo
left a comment
There was a problem hiding this comment.
One blocking memory-boundary concern is noted inline.
| max_history_items: Option<usize>, | ||
| streaming: bool, | ||
| ) -> Result<ResponsesState, FilterAction> { | ||
| let stored = source.stored_messages(); |
There was a problem hiding this comment.
[P1] Enforce the limits before cloning the stored history. source.stored_messages() materializes the entire Vec here, while ConversationRecord/ResponseRecord have already been fully deserialized by get_conversation/get_response; enforce_history_limits therefore runs only after the potentially unbounded allocation has happened. A large accumulated conversation can still cause a substantial memory spike before returning 413, so max_history_bytes does not provide the advertised memory bound. Please enforce the item/byte limits on the borrowed stored JSON (or fetch a bounded representation) before cloning it into the request state.
Summary
Closes #66.
max_history_bytes(default 2 MiB) andmax_history_items(default unlimited) config fields to theopenai_responses_rehydratefilterResponsesState, preventing unbounded proxy memory growthRehydrationSourcetrait sobuild_stateis generic overResponseRecordandConversationRecord— limits are enforced uniformly on both theprevious_response_idandconversationrehydration pathsrehydrateinto a clean dispatcher with two symmetric methods:rehydrate_from_responseandrehydrate_from_conversationusize::MAX(reject) instead of0(allow)Config
Both fields reject zero values at config load time.
Test plan
cargo test -p praxis-ai-apis -- rehydrate)max_history_bytes, zeromax_history_items, unknown fieldsmake lintclean (clippy + nightly fmt + filter docs)