RecursionError + "input length and max_tokens exceed context limit

[gargshilpi22]

#152

Description

[Provide a detailed description of the changes in this PR]

Related Issues

[Link to related issues using #issue-number format]

Documentation PR

[Link to related associated PR in the agent-docs repo]

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Other (please describe):

[Choose one of the above types of changes]

Testing

[How have you tested the change?]

• hatch fmt --linter
• hatch fmt --formatter
• hatch test --all
• Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

Checklist

• I have read the CONTRIBUTING document
• I have added tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature
• My changes generate no new warnings
• Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[gargshilpi22]

This fix solved the following issue: #152

But when I ran a benchmark, I saw another exception in the logs:

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: Expected toolResult blocks at messages.1.content for the following Ids: tooluse_gzsNNUzFQDyh7V7ZH3Dm6g, but found: tooluse_Rv0mh9kdSv6Sc8rzstK7HQ

I wonder why we are not calling the reduce_context when the context limit exception happens in the event loop?

[mrityunjayshukla]

Hi @gargshilpi22, Thanks for raising potential fix PR. I am just ramping up on strands sdk (not part of strands team) hence few followup questions:

1. could aggressive truncation mean losing more conversation context hence could lead to inaccurate result?
2. should we make last N messages truncation configurable by users (similar to window_size in SlidingWindowConversationManager)? Also, should we bring this logic out of error_handler ?

[Unshure]

Hi @gargshilpi22, I wanted to point you to the logic in our Agent class where we are calling reduce_context on the conversation_manager: https://github.com/strands-agents/sdk-python/blob/main/src/strands/agent/agent.py#L506-L509

The reduce_context function is called whenever a model throws a ContextWindowOverflowException, and this should ideally trim down the conversation to avoid this issue. It seems like there might be a bug in this logic. Can you share the state of the agent when you run into this issue? The system_prompt, messages, and tools will help me investigate and reproduce this issue myself.

[mrityunjayshukla]

https://github.com/strands-agents/sdk-python/blob/main/src/strands/agent/agent.py#L506-L509

Hi @Unshure , I think reduce_context and handle_input_too_long_error are used in different scenarios to reduce/handle the ContextWindowOverflow . In context of this issue, the scenario is related to multi-turn tool interactions where recursive calls are made as part of handling ContextWindowOverflowException thrown as result of single conversation where multi tool calls are made(ref). The recursion logic inside handle_input_too_long_error method is causing infinite recursion as per my understanding.