Replies: 4 comments 2 replies
-
The regex looks OK in the code? Line 1096 in ea57e47 I'm confused. |
Beta Was this translation helpful? Give feedback.
-
Oh, the problem is the function name isn't quoted. So anyway, the real question is whether there's a jinja for Llama pythonic syntax? |
Beta Was this translation helpful? Give feedback.
-
Here's the example prompt for watt-tool:
Really weird. They use but bypass most of the chat template they have. I guess it will require a custom jinja. |
Beta Was this translation helpful? Give feedback.
-
Hey @edmcman, thanks for looking into this! It seems the model likes to wrap its tool calls inside wiki block syntax
That's a syntax commonly used in other models's styles (DeepSeek R1, Qwen2.5 Coder) and actually helps disambiguate some outputs, should be straightforward to support. Adding to my todo list, might not get to it before pushing some of the tool diff work tho. (and then we'll be able to just use |
Beta Was this translation helpful? Give feedback.
-
cc @ochafik
Has anyone tested watt-tool-8B with tool calling? watt-tool-8B doesn't seem to have a proper chat template. watt-tool-70B does, but it is the old "JSON" style llama prompt. The how to use section of the README suggests that it needs pythonic style function calls. Is there an official jinja around for that?
I tried watt-tool-8b with the watt-tool-70B jinja, and it generated what looked like valid llama JSON calls to me:
{"name": get_decompiled_function, "parameters": {"binary_hash": "12a6d908a68ccf6f9f3d799705577c28763f5deef6eddcff7643d6d8a6de543d", "function_addr": "0x1101b0"}}
But it doesn't match the following regex:
Is this regex correct? It doesn't look right to me. It doesn't look like it would match this:
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
Beta Was this translation helpful? Give feedback.
All reactions