Limited Extensibility in ChatMemoryRepository Design

[huge0612]

Issue Title: Limited Extensibility in ChatMemoryRepository Design

Description:
In the latest Spring AI 1.0.0 release, the ChatMemory interface and its default MessageWindowChatMemory implementation (using fixed-window conversation history) provide good extensibility for custom implementations. However, there appears to be tight coupling between the storage mechanism and windowed memory implementation through the ChatMemoryRepository interface.

The current ChatMemoryRepository interface:

public interface ChatMemoryRepository {
    List<String> findConversationIds();
    List<Message> findByConversationId(String conversationId);
    void saveAll(String conversationId, List<Message> messages); // Overwrites existing messages
    void deleteByConversationId(String conversationId);
}

Presents two key limitations:

1. Save Mechanism: The saveAll method requires full overwrite of existing messages, preventing incremental storage of conversation history.
2. Query Capability: findByConversationId forces retrieval of all messages, making it impossible to implement partial retrieval patterns (e.g., top-N messages).

This design essentially binds ChatMemoryRepository implementations to work exclusively with the windowed memory approach used by MessageWindowChatMemory. Developers cannot implement alternative strategies like:

• Storing complete conversation history while only loading recent messages
• Implementing custom message retrieval logic (e.g., time-based or relevance-based sampling)

The current architecture forces complete reimplementation of both ChatMemory and storage mechanisms for non-windowed use cases, negating the interface's intended extensibility benefits. Could we reconsider this design to better separate storage concerns from memory management strategies?

Suggested Improvements:

1. Add support for incremental writes (e.g., addMessage method)
2. Allow paginated/parameterized message retrieval (e.g., limit/offset parameters)

[sunyuhan1998]

Actually, I have a similar question myself. From what I understand, the implementation of ChatMemory should involve different storage strategies (e.g., short-term memory, long-term memory, or any others).

The default implementation currently provided, MessageWindowChatMemory, is a strategy that maintains a specified window size. However, the ChatMemoryRepository it calls also seems to be implemented purely based on the same "fixed window size, evicting older messages" strategy. So perhaps it would make more sense for it to be named MessageWindowChatMemoryRepository.

My confusion lies in this: Is the ChatMemoryRepository intended to serve all types of ChatMemory, or is it specific only to MessageWindowChatMemory?

@ThomasVitale , could you please help clarify this for us? We'd greatly appreciate it.

[dqx-eterning]

I really need this feature: "Store the complete conversation history while only loading the most recent messages." In the older version, there seemed to be a similar configuration (CHAT_MEMORY_RETRIEVE_SIZE_KEY), but it has been removed in the latest version.

[huge0612]

It seems that the SpringAI team believes ChatMemory is only suitable for short - term storage. As a result, they don't plan to offer any solutions for long - term storage.

This situation forces developers to implement ChatMemory and ChatHistory separately rather than using a single shared storage. This not only increases the complexity of the development process but may also lead to inefficiencies in data management. It would be beneficial if the SpringAI team could consider the need for long - term storage and explore ways to integrate ChatMemory and ChatHistory more effectively, perhaps by providing some guidelines or design patterns to help developers better handle this aspect of the application.

This would greatly enhance the usability and scalability of the overall system and reduce the burden on developers. We hope to see some positive changes and discussions around this issue in the near future.