Add total token counts to models #1282
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds total token count tracking functionality to the models in the smolagents package. While the existing implementation already tracks last_input_token_count and last_output_token_count for individual calls, this enhancement adds cumulative tracking with total_input_token_count, total_output_token_count, and total_token_count properties to provide users with better visibility into token usage across an entire agent session.
Motivation
Token usage is a critical metric for LLM applications, especially in production environments where costs need to be monitored and optimized. Having access to cumulative token counts allows developers to:
This also enable the use of several models combined (more support of this maybe in future works) to reduce costs depending on the tasks.
Changes
Added total_input_token_count, total_output_token_count properties to base model classes
Initialized these counters to zero in constructors
Added support in the serialization methods
Added update_token_counts method in the base model class
Updated the token counting logic to increment these totals whenever new tokens are processed in all child classes.