Tool Call Improvements

[wendy-aw]

Changes

1. tool_choice schema

Removed pydantic schema for tool_choice. OpenAI and Anthropic have very different inputs for this parameter.

Feature	OpenAI	Anthropic
Zero or more tools called	auto	{"type": "auto"}
At least one tool must be called	required	{"type": "any"}
Specific tool called	{"type": "function", "function": {"name": function_name}}	{"type": "tool", "name": function_name}

We hence simplify things by allowing the input to the chat functions to be simple strings like "auto", "required", "function_name". convert_tool_choice function will then map these to the right formats above for the different models. Also if tools are listed but tool_choice is not set, we default to "auto".

2. Avoid looping with required tool_choice

Previously if tool_choice was set to required, tool chaining would force the model to use a tool in each round. This will continue indefinitely and would not end with a message. We fix this by setting tool_choice to be auto after the first tool call.

3. tool_outputs in LLMResponse

We output the intermediate tool outputs during tool chaining so that we can inspect the steps the model has taken.

4. New tests

We port the tests for tool calls from defog_utils. However, we retain get_weather tests and do away with the web_search tests so that we do not require a defog API key. Other tests have also been added to cases where tool_choice is required or a forced function. Anthropic tests now also use sonnet instead of haiku as it's more reliable.

[rishsriv]

Neat! Thank you for documenting this so thoroughly, and for the added tests. Returning results from tool use is super helpful!

[wongjingping]

very neat, thanks for adding the rigorous battery of tests!