Skip to content

Limited Extensibility in ChatMemoryRepository Design #3288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
huge0612 opened this issue May 22, 2025 · 3 comments
Open

Limited Extensibility in ChatMemoryRepository Design #3288

huge0612 opened this issue May 22, 2025 · 3 comments

Comments

@huge0612
Copy link

Issue Title: Limited Extensibility in ChatMemoryRepository Design

Description:
In the latest Spring AI 1.0.0 release, the ChatMemory interface and its default MessageWindowChatMemory implementation (using fixed-window conversation history) provide good extensibility for custom implementations. However, there appears to be tight coupling between the storage mechanism and windowed memory implementation through the ChatMemoryRepository interface.

The current ChatMemoryRepository interface:

public interface ChatMemoryRepository {
    List<String> findConversationIds();
    List<Message> findByConversationId(String conversationId);
    void saveAll(String conversationId, List<Message> messages); // Overwrites existing messages
    void deleteByConversationId(String conversationId);
}

Presents two key limitations:

  1. Save Mechanism: The saveAll method requires full overwrite of existing messages, preventing incremental storage of conversation history.
  2. Query Capability: findByConversationId forces retrieval of all messages, making it impossible to implement partial retrieval patterns (e.g., top-N messages).

This design essentially binds ChatMemoryRepository implementations to work exclusively with the windowed memory approach used by MessageWindowChatMemory. Developers cannot implement alternative strategies like:

  • Storing complete conversation history while only loading recent messages
  • Implementing custom message retrieval logic (e.g., time-based or relevance-based sampling)

The current architecture forces complete reimplementation of both ChatMemory and storage mechanisms for non-windowed use cases, negating the interface's intended extensibility benefits. Could we reconsider this design to better separate storage concerns from memory management strategies?

Suggested Improvements:

  1. Add support for incremental writes (e.g., addMessage method)
  2. Allow paginated/parameterized message retrieval (e.g., limit/offset parameters)
@huge0612 huge0612 reopened this May 22, 2025
@sunyuhan1998
Copy link
Contributor

Actually, I have a similar question myself. From what I understand, the implementation of ChatMemory should involve different storage strategies (e.g., short-term memory, long-term memory, or any others).

The default implementation currently provided, MessageWindowChatMemory, is a strategy that maintains a specified window size. However, the ChatMemoryRepository it calls also seems to be implemented purely based on the same "fixed window size, evicting older messages" strategy. So perhaps it would make more sense for it to be named MessageWindowChatMemoryRepository.

My confusion lies in this: Is the ChatMemoryRepository intended to serve all types of ChatMemory, or is it specific only to MessageWindowChatMemory?

@ThomasVitale , could you please help clarify this for us? We'd greatly appreciate it.

@dqx-eterning
Copy link

I really need this feature: "Store the complete conversation history while only loading the most recent messages." In the older version, there seemed to be a similar configuration (CHAT_MEMORY_RETRIEVE_SIZE_KEY), but it has been removed in the latest version.

@huge0612
Copy link
Author

It seems that the SpringAI team believes ChatMemory is only suitable for short - term storage. As a result, they don't plan to offer any solutions for long - term storage.

This situation forces developers to implement ChatMemory and ChatHistory separately rather than using a single shared storage. This not only increases the complexity of the development process but may also lead to inefficiencies in data management. It would be beneficial if the SpringAI team could consider the need for long - term storage and explore ways to integrate ChatMemory and ChatHistory more effectively, perhaps by providing some guidelines or design patterns to help developers better handle this aspect of the application.

This would greatly enhance the usability and scalability of the overall system and reduce the burden on developers. We hope to see some positive changes and discussions around this issue in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants