fix(responses): streaming with tool_choice allowed_tools fails #17205
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
fix: streaming with tool_choice allowed_tools fails with pickle error
Relevant issues
Fixes #17192
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hardrequirement - see details
make test-unitType
🐛 Bug Fix
Changes
Summary
When using the Responses API with
stream=Trueandtool_choiceof typeallowed_tools, the streaming fails with:TypeError: cannot pickle 'pydantic_core._pydantic_core.ValidatorIterator' object. In
_handle_logging_completed_response()when callingcopy.deepcopy()on the completed response. Thetool_choicefield contains a PydanticValidatorIteratorthat cannot be serialized with pickle.Solution
Replace
copy.deepcopy()with Pydantic'smodel_dump()+model_validate()pattern in both async and sync_handle_logging_completed_response()methods.Files Changed
litellm/responses/streaming_iterator.pytests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.pyTest passing
======================== 6 passed, 9 warnings in 0.09s =========================