Skip to content

RecursionError + "input length and max_tokens exceed context limit #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

RecursionError + "input length and max_tokens exceed context limit #164

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion src/strands/event_loop/error_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def handle_throttling_error(
current_delay,
max_attempts,
attempt + 1,
)
)
callback_handler(event_loop_throttled_delay=current_delay, **kwargs)
time.sleep(current_delay)
new_delay = min(current_delay * 2, max_delay) # Double delay each retry
Expand Down Expand Up @@ -95,6 +95,16 @@ def handle_input_too_long_error(
"""
from .event_loop import recurse_event_loop # Import here to avoid circular imports

# Check for recursion depth to prevent infinite recursion
recursion_depth = kwargs.get("_truncation_recursion_depth", 0)
if recursion_depth >= 5: # Limit recursion to 5 levels
callback_handler(force_stop=True, force_stop_reason="Maximum truncation attempts reached. Input is still too long.")
logger.error("Maximum truncation recursion depth reached. Input is still too long.")
raise ContextWindowOverflowException("Maximum truncation attempts reached. Input is still too long.") from e

# Increment recursion depth counter
kwargs["_truncation_recursion_depth"] = recursion_depth + 1

# Find the last message with tool results
last_message_with_tool_results = find_last_message_with_tool_results(messages)

Expand All @@ -105,6 +115,16 @@ def handle_input_too_long_error(
# Truncate the tool results in this message
truncate_tool_results(messages, last_message_with_tool_results)

# If we're at a high recursion depth, be more aggressive with truncation
if recursion_depth >= 2:
# Remove older messages if there are more than a few
if len(messages) > 4:
# Keep the first message (usually system) and the last 3 messages
preserved_messages = [messages[0]] + messages[-3:]
messages.clear()
messages.extend(preserved_messages)
logger.debug("Aggressively truncated conversation history to %s messages", len(messages))
Comment on lines +119 to +126
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @gargshilpi22, Thanks for raising potential fix PR. I am just ramping up on strands sdk (not part of strands team) hence few followup questions:

  1. could aggressive truncation mean losing more conversation context hence could lead to inaccurate result?
  2. should we make last N messages truncation configurable by users (similar to window_size in SlidingWindowConversationManager)? Also, should we bring this logic out of error_handler ?


return recurse_event_loop(
model=model,
system_prompt=system_prompt,
Expand Down