community: have ChatLlamaCpp handle "auto" and "any" for tool_choice #30810
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
dosubot
bot
added
size:XS
| @@ -362,6 +363,8 @@ def bind_tools( | |||
| f"Tool choice {tool_choice=} was specified, but the only " | |||
| f"provided tools were {tool_names}." | |||
| ) | |||
| elif isinstance(tool_choice, str) and tool_choice in ['any', 'auto']: | |||
| @@ -362,6 +363,8 @@ def bind_tools( | |||
| f"Tool choice {tool_choice=} was specified, but the only " | |||
| f"provided tools were {tool_names}." | |||
| ) | |||
| elif isinstance(tool_choice, str) and tool_choice in ['any', 'auto']: | |||
| tool_choice = formatted_tools[0] | |||
There was a problem hiding this comment.
IMO, if tools is length one, then we should translate "any" as you do here, but if there are multiple tools, we should raise an error.
clinty
force-pushed
the
llamacpp-auto-any
branch
from
2adf9ab to
2df023f
Compare
dosubot
bot
added
size:S
|
@ccurme what about this? I suppose it does the wrong thing if there's a tool called "auto". |
There was a problem hiding this comment.
I think "auto" should have the same behavior as None for tool choice, right? Should we just assign None in that case?
For "any", my thought is to do something like this:
if tool_choice == "any":
if len(formatted_tools) == 1:
tool_choice = formatted_tools[0]
else:
raise ValueError(
"tool_choice `'any'` only supported if one tool is provided."
)This change will need some tests as well.
clinty
force-pushed
the
llamacpp-auto-any
branch
from
2df023f to
58fd626
Compare
dosubot
bot
added
size:M
|
@ccurme where should the tests go? |
There was a problem hiding this comment.
@clinty we've now moved langchain-community into a standalone repo. Would you mind opening your PR there with a test? Feel free to tag me.
Description: This adds "auto" and "any" to ChatLlamaCpp's tool_choice options, and handles them both by selecting the first tool in the list
Issue: trustcall assumes support for "any" at https://github.com/AIXerum/trustcall/blob/main/trustcall/_base.py#L717