Skip to content

[FEATURE] In-envent-loop cycle context management #298

Open
@RingoIngo2

Description

@RingoIngo2

Problem Statement

The ConversationManager is executed after a complete cycle, however, even within a cycle, context management would be a great benefit.

Consider the following use case:

  • Within one cycle, the agent makes multiple tool calls.
  • The output of these tool calls is large (i.e., many tokens).
  • Tool results k steps back in the past (within the cycle) don't matter anymore for deciding the next tool call.

In this scenario, we would like to manage the context by truncating older (k steps in the past) tool results for increased latency, decreased cost, or in the extreme case, to avoid throttling.

Proposed Solution

This could perhaps be an additional function on the conversation manager interface?
Or perhaps this should be a new/custom conversation manager?

What about the following:

  • We add a property manage_in_cycle to ConversationManager
  • In Agent._execute_event_loop_cycle, we pass conversation_manager to event_loop_cycle
  • we pass it on to _handle_tool_execution, then to recurse_event_loop
  • Somewhere there, we execute conversation_manager.apply_management conditioned on conversation_manager.manage_in_cycle being True.

Use Case

  • Within one cycle, the agent makes multiple tool calls.
  • The output of these tool calls is large (i.e., many tokens).
  • Tool results k steps back in the past (within the cycle) don't matter anymore for deciding the next tool call.

In this scenario, we would like to manage the context by truncating older (k steps in the past) tool results for increased latency, decreased cost, or in the extreme case, to avoid throttling.

Alternatives Solutions

No response

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions