Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tool choice docs #349

Merged
merged 5 commits into from
Jan 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions fern/pages/text-generation/tools/tool-use.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,18 @@ if not response.tool_calls:
co.chat(message=message, tools=tools, tool_results=[])
```

## Forcing Tool Use

During the tool calling step, the model may decide to either:
- make tool call(s)
- or, respond to a user message directly.

You can force the model to make tool call(s), i.e. to not respond directly, by setting the `force_single_step=True` and providing some tool definitions through the `tools` parameter.

This is equivalent to setting the `tool_choice` as `REQUIRED` in the v2 API.

Besides, you can force the model to respond directly, by setting `force_single_step=True` and by providing some tool results through the `tool_results` parameter. This is equivalent to specifying `tool_choice` as `NONE` in the v2 API.

## Single-Step Tool Use and Chat History

Single-step tool use functions as part of a two-part conversational process. Here’s how that works:
Expand Down
36 changes: 31 additions & 5 deletions fern/pages/v2/text-generation/tools/tool-use.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -411,17 +411,43 @@ Start: 177 | End: 204 | Text: 'Laptop: $1,000, 15 in stock'
Start: 207 | End: 232 | Text: 'Tablet: $300, 25 in stock'
```

## Forcing Tool Use

During the tool calling step, the model may decide to either:
- make tool call(s)
- or, respond to a user message directly.

You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`.

Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`.

By default, if you don’t specify the `tool_choice` parameter, then the model will decide whether it's more appropriate to call tools or to respond directly.

```python PYTHON {5}
response = co.chat(
model="command-r-plus-08-2024",
messages=messages,
tools=tools,
tool_choice="REQUIRED" # optional, to force tool calls
# tool_choice="NONE" # optional, to force a direct response
)
```

<Note>This parameter is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models.</Note>

## Structured Outputs (Tools)

Setting the `strict_tools` parameter to `True` will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the [Structured Outputs documentation](https://docs.cohere.com/v2/docs/structured-outputs).

Note that `strict_tools` is currently an experimental feature.

```python PYTHON {4}
response = co.chat(model="command-r-plus-08-2024",
messages=messages,
tools=tools,
strict_tools=True)
```python PYTHON {5}
response = co.chat(
model="command-r-plus-08-2024",
messages=messages,
tools=tools,
strict_tools=True
)
```

## How to Get Good Answers With Tool Use
Expand Down
Loading