You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The addition of a ConversationManager that can summarize and replace the conversation thus far. Currently, when faced with a ContextWindowOverflowError, the Agent can be configured with a SlidingWindowConversationManager that trims off the oldest messages in the conversation until it fits into the models context window. This leads to the deletion of context from the conversation.
Proposed Solution
When trying to reduce the context of the conversation history, instead of removing context all-together, instead the Agent can use an LLM to summarize the conversation thus-far, and then use the summarization to replace the history.
Create a new CompactionConversationManager class
Implement the reduce_context function that will call an agent with the compaction prompt and the current messages array. It will overwrite the messages history with the summarized version.
Have this ConversationManager use the parent Agent model_provider by default, but allow for an optional system_prompt override.
The new ConversationManager can also take in an Agent at initialization time that would be used for summarization instead of the default parent Agent. This allows for the specification of a summarization agent that can use tools as part of the summarization process.
Considerations:
This will need to update reduce_context abstract method to pass in an agent: Agent parameter
Tools should be able to be used by the summarization agent override
If the messages array is too large to summarize, break it in half by character count, and summarize each part
Use Case
If the messages array overflows during conversation, the conversation is summarized and can continue as normal.
Alternatives Solutions
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered:
@Unshure I would find this very useful and agree with this approach. A few comments:
Nitpick: SummarizationConversationManager/SummarizingConversationManager naming sounds more descriptive vs CompactionConversationManager (no strong preference here, just a thought);
Do you think it would be useful for the conversation manager to accept a compression ratio and a minimum number of recent messages to retain, or not relevant?
// summary_ratio: Ratio of messages to summarize vs keep when the window is exceeded. ie. if set to 0.3 (summarize 30% of the oldest messages).
// preserve_recent_messages: Minimum number of recent messages to always keep.
Overall I'd happy to update the open PR #112 with this proposed implementation.
I agree, both of the names you suggested are more descriptive. I'm fine with either of those.
I think these are useful configurations to help adjust the summarization. Lets make sure that the preserve_recent_messages take precedence over the summary_ratio.
Ill leave some comments on your pull request as well so we can reduce the review-iterations needed. Thanks again for helping with this!
Uh oh!
There was an error while loading. Please reload this page.
Problem Statement
The addition of a ConversationManager that can summarize and replace the conversation thus far. Currently, when faced with a ContextWindowOverflowError, the Agent can be configured with a SlidingWindowConversationManager that trims off the oldest messages in the conversation until it fits into the models context window. This leads to the deletion of context from the conversation.
Proposed Solution
When trying to reduce the context of the conversation history, instead of removing context all-together, instead the Agent can use an LLM to summarize the conversation thus-far, and then use the summarization to replace the history.
CompactionConversationManager
classreduce_context
function that will call an agent with the compaction prompt and the current messages array. It will overwrite the messages history with the summarized version.ConversationManager
use the parentAgent
model_provider by default, but allow for an optionalsystem_prompt
override.ConversationManager
can also take in anAgent
at initialization time that would be used for summarization instead of the default parent Agent. This allows for the specification of a summarization agent that can use tools as part of the summarization process.Considerations:
reduce_context
abstract method to pass in anagent: Agent
parameterUse Case
Alternatives Solutions
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered: