What is the method for fine-tuning an LLM with function calling for a Langchain Agent? Or, how can I build a custom dataset for fine-tuning an LLM with function calling for a Langchain Agent?

[khw11044]

"I built a Langchain agent using OpenAI, and the agent effectively utilizes the tools and provides the desired results according to my intentions. Now, I want to build a Langchain agent using an open-source local LLM. When I built an agent using llama3.2, it doesn't use the tools well. Therefore, I have been diligently researching under the title 'fine tuning llm for function calling.' However, when I actually apply this to the Langchain agent I built, it doesn't work well."

for example,

from langchain_ollama import ChatOllama
from langchain_openai import ChatOpenAI

from langchain.agents import create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import AgentExecutor
from typing import List

from langchain.agents import tool

from dotenv import load_dotenv

API 키 정보 로드

load_dotenv()

@tool
def add(xy_pairs: List[tuple]) -> List[dict]:
"""Add two integers together.
"""
results = []
for x, y in xy_pairs:
result = {
f"{x}+{y}": x + y,
}
results.append(result)
return results

@tool
def multiply(xy_pairs: List[tuple]) -> List[dict]:
"""Multiply two integers together.
"""
results = []
for x, y in xy_pairs:
result = {
f"{x}*{y}": x * y,
}
results.append(result)
return results

tools = [add, multiply]

prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant. "
),
("placeholder", "{chat_history}"),
("human", "{input}"),
("placeholder", "{agent_scratchpad}"),
]
)

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.6)

llm_with_tools = llm.bind_tools(tools)

agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
agent=agent,
tools=tools,
verbose=True,
)

query = "What is 3 * 12?"

result = agent_executor.invoke({"input": query})
log is

Entering new AgentExecutor chain...

Invoking: multiply with {'xy_pairs': [[3, 12]]}

[{'3*12': 36}]3 * 12 is 36.

Finished chain.
and also

log_history = []
stream_iterator = agent_executor.stream({"input": query})
for step in stream_iterator:
log_history.append(step)

log_history
You can see results like the ones below

[{'actions': [ToolAgentAction(tool='multiply', tool_input={'xy_pairs': [[3, 12]]}, log="\nInvoking: multiply with ~
'messages': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call ~
{'steps': [AgentStep(action=ToolAgentAction(tool='multiply', tool_input={'xy_pairs': [[3, 12]]}, log="\nInvoking ~
'messages': [FunctionMessage(content='[{"3*12": 36}]', additional_kwargs={}, response ~
{'output': 'The result of (3 \times 12) is (36).',
'messages': [AIMessage(content='The result of (3 \times ~

However, when using a fine-tuned LLM, I have seen the following results.

Entering new AgentExecutor chain...
{"type":"function","function":{"name":"multiply","arguments":[{"x":3, "y":12}]}}
The result of the multiplication operation between 3 and 12 is 36.

Finished chain.

log_history

{'output': '{"type":"function","function":{"name":"multiply","arguments":[{"x":3, "y":12}]}}\nThe result of the multiplication operation between 3 and 12 is 36.',
'messages': [AIMessage(content='{"type":"function","function":{"name":"multiply","arguments":[{"x":3, "y":12}]}}\nThe result of the multiplication operation between 3 and 12 is 36.', additional_kwargs={}, response_metadata={})]}]

I want to know in detail about building a dataset for a Langchain agent from scratch. I want to fine-tune a local small LLM so that it can follow the same flow as the trajectories of OpenAI's Langchain agent as much as possible. How should I do this?

[dosubot[bot]]

Hello @khw11044! I'm here to help you with any questions or issues you have while you wait for a human maintainer. Let’s tackle this together!

To build a custom dataset for fine-tuning a local small LLM with function calling to create a Langchain agent that follows the same flow as OpenAI's Langchain agent, you can follow these steps:

1. Install Prerequisites: Ensure you have installed langchain >= 0.0.311 and have configured your environment with your LangSmith API key.

   %pip install --upgrade --quiet langchain langchain-openai

2. Prepare the Data: Use the LangSmithDatasetChatLoader to load chat sessions from successful LLM runs. Convert these sessions into a format suitable for fine-tuning using convert_messages_for_finetuning.

   import os
   import uuid
   from langsmith.client import Client
   from langchain_community.chat_loaders.langsmith import LangSmithDatasetChatLoader
   from langchain_community.adapters.openai import convert_messages_for_finetuning
   
   uid = uuid.uuid4().hex[:6]
   os.environ["LANGSMITH_TRACING"] = "true"
   os.environ["LANGSMITH_API_KEY"] = "YOUR API KEY"
   
   client = Client()
   url = "https://raw.githubusercontent.com/langchain-ai/langchain/master/docs/docs/integrations/chat_loaders/example_data/langsmith_chat_dataset.json"
   response = requests.get(url)
   response.raise_for_status()
   data = response.json()
   
   dataset_name = f"Extraction Fine-tuning Dataset {uid}"
   ds = client.create_dataset(dataset_name=dataset_name, data_type="chat")
   
   _ = client.create_examples(
       inputs=[e["inputs"] for e in data],
       outputs=[e["outputs"] for e in data],
       dataset_id=ds.id,
   )
   
   loader = LangSmithDatasetChatLoader(dataset_name=dataset_name)
   chat_sessions = loader.lazy_load()
   training_data = convert_messages_for_finetuning(chat_sessions)

3. Fine-tune the Model: Use the OpenAI library to fine-tune the model with the prepared training data.

   import json
   import time
   from io import BytesIO
   import openai
   
   my_file = BytesIO()
   for dialog in training_data:
       my_file.write((json.dumps({"messages": dialog}) + "\n").encode("utf-8"))
   
   my_file.seek(0)
   training_file = openai.files.create(file=my_file, purpose="fine-tune")
   
   job = openai.fine_tuning.jobs.create(
       training_file=training_file.id,
       model="gpt-3.5-turbo",
   )
   
   # Wait for the fine-tuning to complete
   status = openai.fine_tuning.jobs.retrieve(job.id).status
   start_time = time.time()
   while status != "succeeded":
       print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
       time.sleep(5)
       status = openai.fine_tuning.jobs.retrieve(job.id).status

4. Use the Fine-tuned Model in LangChain: After fine-tuning, use the resulting model ID with the ChatOpenAI model class in your LangChain app.

   # Get the fine-tuned model ID
   job = openai.fine_tuning.jobs.retrieve(job.id)
   model_id = job.fine_tuned_model
   
   # Use the fine-tuned model in LangChain
   from langchain_openai import ChatOpenAI
   
   model = ChatOpenAI(
       model=model_id,
       temperature=1,
   )
   
   model.invoke("There were three ravens sat on a tree.")

5. Run a Local LLM Server: If you are using a local LLM, start the model server. For example, using TinyLlama:

   ./TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile --server --nobrowser

6. Interact with the Local Model: Use LangChain to interact with your local model.

   from langchain_community.llms.llamafile import Llamafile
   
   llm = Llamafile()
   
   llm.invoke("The first man on the moon was ... Let's think step by step.")

This process allows you to create a custom dataset and fine-tune a model to follow the same flow as OpenAI's Langchain agent, while also supporting local LLMs ^[1][2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}