Skip to content

[FEATURE] Compaction Conversation Manager #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Unshure opened this issue May 23, 2025 · 2 comments · May be fixed by #112 or strands-agents/docs#63
Open

[FEATURE] Compaction Conversation Manager #90

Unshure opened this issue May 23, 2025 · 2 comments · May be fixed by #112 or strands-agents/docs#63
Assignees
Labels
enhancement New feature or request

Comments

@Unshure
Copy link
Member

Unshure commented May 23, 2025

Problem Statement

The addition of a ConversationManager that can summarize and replace the conversation thus far. Currently, when faced with a ContextWindowOverflowError, the Agent can be configured with a SlidingWindowConversationManager that trims off the oldest messages in the conversation until it fits into the models context window. This leads to the deletion of context from the conversation.

Proposed Solution

When trying to reduce the context of the conversation history, instead of removing context all-together, instead the Agent can use an LLM to summarize the conversation thus-far, and then use the summarization to replace the history.

  1. Create a new CompactionConversationManager class
  2. Implement the reduce_context function that will call an agent with the compaction prompt and the current messages array. It will overwrite the messages history with the summarized version.
  3. Have this ConversationManager use the parent Agent model_provider by default, but allow for an optional system_prompt override.
  4. The new ConversationManager can also take in an Agent at initialization time that would be used for summarization instead of the default parent Agent. This allows for the specification of a summarization agent that can use tools as part of the summarization process.

Considerations:

  • This will need to update reduce_context abstract method to pass in an agent: Agent parameter
  • Tools should be able to be used by the summarization agent override
  • If the messages array is too large to summarize, break it in half by character count, and summarize each part

Use Case

  • If the messages array overflows during conversation, the conversation is summarized and can continue as normal.

Alternatives Solutions

No response

Additional Context

No response

@Unshure Unshure added the enhancement New feature or request label May 23, 2025
@Unshure Unshure marked this as a duplicate of #111 May 26, 2025
@stefanoamorelli
Copy link

@Unshure I would find this very useful and agree with this approach. A few comments:

  1. Nitpick: SummarizationConversationManager/SummarizingConversationManager naming sounds more descriptive vs CompactionConversationManager (no strong preference here, just a thought);
  2. Do you think it would be useful for the conversation manager to accept a compression ratio and a minimum number of recent messages to retain, or not relevant?
// summary_ratio: Ratio of messages to summarize vs keep when the window is exceeded. ie. if set to 0.3 (summarize 30% of the oldest messages).
// preserve_recent_messages: Minimum number of recent messages to always keep.

Overall I'd happy to update the open PR #112 with this proposed implementation.

@Unshure
Copy link
Member Author

Unshure commented May 26, 2025

@stefanoamorelli, to address your comments:

  1. I agree, both of the names you suggested are more descriptive. I'm fine with either of those.
  2. I think these are useful configurations to help adjust the summarization. Lets make sure that the preserve_recent_messages take precedence over the summary_ratio.

Ill leave some comments on your pull request as well so we can reduce the review-iterations needed. Thanks again for helping with this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
3 participants