Open
Description
Problem Statement
The ConversationManager
is executed after a complete cycle, however, even within a cycle, context management would be a great benefit.
Consider the following use case:
- Within one cycle, the agent makes multiple tool calls.
- The output of these tool calls is large (i.e., many tokens).
- Tool results k steps back in the past (within the cycle) don't matter anymore for deciding the next tool call.
In this scenario, we would like to manage the context by truncating older (k steps in the past) tool results for increased latency, decreased cost, or in the extreme case, to avoid throttling.
Proposed Solution
This could perhaps be an additional function on the conversation manager interface?
Or perhaps this should be a new/custom conversation manager?
What about the following:
- We add a property
manage_in_cycle
toConversationManager
- In
Agent._execute_event_loop_cycle
, we passconversation_manager
toevent_loop_cycle
- we pass it on to
_handle_tool_execution
, then torecurse_event_loop
- Somewhere there, we execute
conversation_manager.apply_management
conditioned onconversation_manager.manage_in_cycle
being True.
Use Case
- Within one cycle, the agent makes multiple tool calls.
- The output of these tool calls is large (i.e., many tokens).
- Tool results k steps back in the past (within the cycle) don't matter anymore for deciding the next tool call.
In this scenario, we would like to manage the context by truncating older (k steps in the past) tool results for increased latency, decreased cost, or in the extreme case, to avoid throttling.
Alternatives Solutions
No response
Additional Context
No response