You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I’m implementing a RAG system with Spring AI that combines document-based context with dynamic, real-time data fetched via tools. However, I’ve encountered an issue where combining RetrievalAugmentationAdvisor with ToolCallbacks leads to significantly reduced tool usage by the model.
Use Case:
My application includes a knowledge base of static documents. These documents contain valuable information (e.g., the description of an employee’s role responsibilities and their name at the time of writing), but some data (e.g., the current employee's with that role name) must be retrieved at runtime via tools.
When I run ChatClient using only RetrievalAugmentationAdvisor or only ToolCallbacks, each works correctly. However, when both are enabled toghether, the LLM strongly prefers the retrieval context and rarely calls tools - but sometimes it does, so it is aware of them.
I attempted to mitigate this by adjusting the system prompt and modifying the ContextualQueryAugmenter’s prompt template to explicitly encourage tool usage over static context when appropriate, but this had no noticeable effect.
Models Tested:
gpt-4.1
gemini-2.5-flash-preview-04-17
gpt-4.1-mini (which seemed slightly more inclined to use tools)
Question/Issue:
I am not sure that this is a bug of the RetrievalAugmentationAdvisor implementation in Spring AI, or rather a limitation of used LLMs.
Is there any guidance on how can RetrievalAugmentationAdvisor be effectively complemented by tools providing fresher data?
The text was updated successfully, but these errors were encountered:
I’m implementing a RAG system with Spring AI that combines document-based context with dynamic, real-time data fetched via tools. However, I’ve encountered an issue where combining
RetrievalAugmentationAdvisor
withToolCallbacks
leads to significantly reduced tool usage by the model.Use Case:
My application includes a knowledge base of static documents. These documents contain valuable information (e.g., the description of an employee’s role responsibilities and their name at the time of writing), but some data (e.g., the current employee's with that role name) must be retrieved at runtime via tools.
When I run
ChatClient
using onlyRetrievalAugmentationAdvisor
or onlyToolCallbacks
, each works correctly. However, when both are enabled toghether, the LLM strongly prefers the retrieval context and rarely calls tools - but sometimes it does, so it is aware of them.I attempted to mitigate this by adjusting the system prompt and modifying the
ContextualQueryAugmenter
’s prompt template to explicitly encourage tool usage over static context when appropriate, but this had no noticeable effect.Models Tested:
gpt-4.1
gemini-2.5-flash-preview-04-17
gpt-4.1-mini
(which seemed slightly more inclined to use tools)Question/Issue:
I am not sure that this is a bug of the
RetrievalAugmentationAdvisor
implementation in Spring AI, or rather a limitation of used LLMs.Is there any guidance on how can
RetrievalAugmentationAdvisor
be effectively complemented by tools providing fresher data?The text was updated successfully, but these errors were encountered: