tutorials/43_Building_a_Tool_Calling_Agent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2OvkPji9O-qX"
   },
   "source": [
    "# Tutorial: Building a Tool-Calling Agent\n",
    "\n",
    "- **Level**: Beginner\n",
    "- **Time to complete**: 15 minutes\n",
    "- **Components Used**: [`Agent`](https://docs.haystack.deepset.ai/docs/agent), [`OpenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [`SerperDevWebSearch`](https://docs.haystack.deepset.ai/docs/serperdevwebsearch), [`ComponentTool`](https://docs.haystack.deepset.ai/docs/componenttool), [`SuperComponent`](https://docs.haystack.deepset.ai/docs/supercomponents)\n",
    "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and a [SerperDev API Key](https://serper.dev/api-key)\n",
    "- **Goal**: After completing this tutorial, you'll have learned how to create an Agent that can use tools both components and pipelines to answer questions and perform tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LFqHcXYPO-qZ"
   },
   "source": [
    "## Overview\n",
    "\n",
    "In this tutorial, you'll learn how to create an agent that can use tools to answer questions and perform tasks. We'll explore two approaches:\n",
    "\n",
    "1. Using the `Agent` with a simple web search tool\n",
    "2. Using the `Agent` with a more complex pipeline with multiple components\n",
    "\n",
    "The `Agent` component allows you to create AI assistants that can use tools to gather information, perform actions, and interact with external systems. It uses a large language model (LLM) to understand user queries and decide which tools to use to answer them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QXjVlbPiO-qZ"
   },
   "source": [
    "## Preparing the Environment\n",
    "\n",
    "First, let's install Haystack and two other dependencies we'll need later:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UQbU8GUfO-qZ"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "pip install haystack-ai docstring-parser trafilatura"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_lvfew16O-qa"
   },
   "source": [
    "### Enter API Keys\n",
    "\n",
    "Enter your API keys for OpenAI and SerperDev:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CbVN-s5LO-qa"
   },
   "outputs": [],
   "source": [
    "from getpass import getpass\n",
    "import os\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")\n",
    "if \"SERPERDEV_API_KEY\" not in os.environ:\n",
    "    os.environ[\"SERPERDEV_API_KEY\"] = getpass(\"Enter SerperDev API key: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yL8nuJdWO-qa"
   },
   "source": [
    "## Using Agent with a Component as a Tool\n",
    "\n",
    "We start with a simple example of using the Agent as a standalone component with a web search tool. The tool can trigger web searches and fetch the search engine results page (SERP) containing the most relevant search hits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XvLVaFHTO-qb"
   },
   "outputs": [],
   "source": [
    "from haystack.components.agents import Agent\n",
    "from haystack.components.generators.chat import OpenAIChatGenerator\n",
    "from haystack.components.websearch import SerperDevWebSearch\n",
    "from haystack.dataclasses import ChatMessage\n",
    "from haystack.tools.component_tool import ComponentTool\n",
    "\n",
    "# Create a web search tool using SerperDevWebSearch\n",
    "web_tool = ComponentTool(component=SerperDevWebSearch(), name=\"web_tool\")\n",
    "\n",
    "# Create the agent with the web search tool\n",
    "agent = Agent(chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"), tools=[web_tool])\n",
    "\n",
    "# Run the agent with a query\n",
    "result = agent.run(messages=[ChatMessage.from_user(\"Find information about Haystack AI framework\")])\n",
    "\n",
    "# Print the final response\n",
    "print(result[\"messages\"][-1].text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RkaQjJJX0FAU"
   },
   "source": [
    "\n",
    "The Agent has a couple of optional parameters that let you customize it's behavior:\n",
    "- `system_prompt` for defining a system prompt with instructions for the Agent's LLM\n",
    "- `exit_conditions` that will cause the agent to return. It's a list of strings and the items can be `\"text\"`, which means that the Agent will exit as soon as the LLM replies only with a text response,\n",
    "or specific tool names, which make the Agent return right after a tool with that name was called.\n",
    "- `state_schema` for the State that is shared across one agent invocation run. It defines extra information – such as documents or context – that tools can read from or write to during execution. You can use this schema to pass parameters that tools can both produce and consume.\n",
    "- `streaming_callback` to stream the tokens from the LLM directly to output.\n",
    "- `raise_on_tool_invocation_failure` to decide if the agent should raise an exception when a tool invocation fails. If set to False, the exception will be turned into a chat message and passed to the LLM. It can then try to improve with the next tool invocation.\n",
    "- `max_agent_steps` to limit how many times the Agent can call tools and prevent endless loops.\n",
    "\n",
    "When `exit_conditions` is set to the default [\"text\"], you can enable streaming so that we see the tokens of the response while they are being generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "aNzUi4iz0FAU"
   },
   "outputs": [],
   "source": [
    "from haystack.components.generators.utils import print_streaming_chunk\n",
    "\n",
    "agent = Agent(\n",
    "    chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"), tools=[web_tool], streaming_callback=print_streaming_chunk\n",
    ")\n",
    "\n",
    "result = agent.run(messages=[ChatMessage.from_user(\"Find information about Haystack AI framework\")])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZnuXEecr0FAU"
   },
   "source": [
    "You can easily switch out the ChatGenerator used in the Agent. Currently all of the following ChatGenerators support tools and thus can be used with Agent:\n",
    "\n",
    "- [AmazonBedrockChatGenerator](https://docs.haystack.deepset.ai/docs/amazonbedrockchatgenerator)\n",
    "- [AnthropicChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicchatgenerator)\n",
    "- [AzureOpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/azureopenaichatgenerator)\n",
    "- [CohereChatGenerator](https://docs.haystack.deepset.ai/docs/coherechatgenerator)\n",
    "- [GoogleAIGeminiChatGenerator](https://docs.haystack.deepset.ai/docs/googleaigeminichatgenerator)\n",
    "- [HuggingFaceAPIChatGenerator](https://docs.haystack.deepset.ai/docs/huggingfaceapichatgenerator)\n",
    "- [HuggingFaceLocalChatGenerator](https://docs.haystack.deepset.ai/docs/huggingfacelocalchatgenerator)\n",
    "- [MistralChatGenerator](https://docs.haystack.deepset.ai/docs/mistralchatgenerator)\n",
    "- [OllamaChatGenerator](https://docs.haystack.deepset.ai/docs/ollamachatgenerator)\n",
    "- [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n",
    "- [VertexAIGeminiChatGenerator](https://docs.haystack.deepset.ai/docs/vertexaigeminichatgenerator)\n",
    "\n",
    "For example, if you have a `HF_API_TOKEN` and `huggingface_hub[inference]>=0.27.0` installed, all you need to do is replace OpenAIChatGenerator by HuggingFaceAPIChatGenerator and run `from haystack.components.generators.chat import HuggingFaceAPIChatGenerator`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HryYZP9ZO-qb"
   },
   "source": [
    "## Using Agent with a Pipeline as Tool\n",
    "\n",
    "Now, for a more sophisticated example, let's build a research assistant that can search the web, fetch content from links, and generate comprehensive answers. In contrast to our previous Agent, we now want to follow the links on the search engine results page, access their content and parse their content through [OutputAdapter](https://docs.haystack.deepset.ai/docs/outputadapter). We'll start with a Haystack Pipeline that the Agent can use as a tool:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "INdC3WvLO-qb",
    "outputId": "fb97b094-b86f-49ca-8c6e-a490134be62f"
   },
   "outputs": [],
   "source": [
    "from haystack.components.builders.answer_builder import AnswerBuilder\n",
    "from haystack.components.converters.html import HTMLToDocument\n",
    "from haystack.components.converters.output_adapter import OutputAdapter\n",
    "from haystack.components.fetchers.link_content import LinkContentFetcher\n",
    "from haystack.components.websearch.serper_dev import SerperDevWebSearch\n",
    "from haystack.dataclasses import ChatMessage\n",
    "from haystack.core.pipeline import Pipeline\n",
    "\n",
    "search_pipeline = Pipeline()\n",
    "\n",
    "search_pipeline.add_component(\"search\", SerperDevWebSearch(top_k=10))\n",
    "search_pipeline.add_component(\"fetcher\", LinkContentFetcher(timeout=3, raise_on_failure=False, retry_attempts=2))\n",
    "search_pipeline.add_component(\"converter\", HTMLToDocument())\n",
    "search_pipeline.add_component(\n",
    "    \"output_adapter\",\n",
    "    OutputAdapter(\n",
    "        template=\"\"\"\n",
    "{%- for doc in docs -%}\n",
    "  {%- if doc.content -%}\n",
    "  <search-result url=\"{{ doc.meta.url }}\">\n",
    "  {{ doc.content|truncate(25000) }}\n",
    "  </search-result>\n",
    "  {%- endif -%}\n",
    "{%- endfor -%}\n",
    "\"\"\",\n",
    "        output_type=str,\n",
    "    ),\n",
    ")\n",
    "\n",
    "search_pipeline.connect(\"search.links\", \"fetcher.urls\")\n",
    "search_pipeline.connect(\"fetcher.streams\", \"converter.sources\")\n",
    "search_pipeline.connect(\"converter.documents\", \"output_adapter.docs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wLIcnWl-66QA"
   },
   "source": [
    "### Creating a Tool from a Pipeline\n",
    "\n",
    "Next, wrap the `search_pipeline` inside a [`SuperComponent`](https://docs.haystack.deepset.ai/docs/supercomponents) and turn it into a tool using `ComponentTool`. The `ComponentTool` automatically creates LLM-compatible tool schemas based on the component’s input sockets. \n",
    "\n",
    "To control what data the `ComponentTool` should receive and returns, you can optionally define `input_mapping` and `output_mapping`. For example, this lets you ensure that only the `\"query\"` input of the `search_pipeline` is mentioned in LLM-compatible tool schema, and only `\"search_result\"` is returned from the `SuperComponent`.\n",
    "\n",
    "Finally, you can initialize the Agent with the resulting `search_tool`.\n",
    "\n",
    "> 💡 Learn alternative ways of creating tools in [`Tool`](https://docs.haystack.deepset.ai/docs/tool) and [`MCPTool`](https://docs.haystack.deepset.ai/docs/mcptool) documentation pages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yxaN3KBo65pv"
   },
   "outputs": [],
   "source": [
    "from haystack.core.super_component import SuperComponent\n",
    "from haystack.tools import ComponentTool\n",
    "from haystack.components.agents import Agent\n",
    "from haystack.components.generators.chat import OpenAIChatGenerator\n",
    "\n",
    "search_component = SuperComponent(\n",
    "    pipeline=search_pipeline,\n",
    "    input_mapping={\"query\": [\"search.query\"]},\n",
    "    output_mapping={\"output_adapter.output\": \"search_result\"},\n",
    ")\n",
    "\n",
    "search_tool = ComponentTool(\n",
    "    name=\"search\",\n",
    "    description=\"Use this tool to search for information on the internet.\",\n",
    "    component=search_component,\n",
    "    outputs_to_string={\"source\": \"search_result\"},\n",
    ")\n",
    "\n",
    "agent = Agent(\n",
    "    chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n",
    "    tools=[search_tool],\n",
    "    system_prompt=\"\"\"\n",
    "    You are a deep research assistant.\n",
    "    You create comprehensive research reports to answer the user's questions.\n",
    "    You use the 'search'-tool to answer any questions.\n",
    "    You perform multiple searches until you have the information you need to answer the question.\n",
    "    Make sure you research different aspects of the question.\n",
    "    Use markdown to format your response.\n",
    "    When you use information from the websearch results, cite your sources using markdown links.\n",
    "    It is important that you cite accurately.\n",
    "    \"\"\",\n",
    "    exit_conditions=[\"text\"],\n",
    "    max_agent_steps=20,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xtORWP3_0FAU"
   },
   "source": [
    "\n",
    "Our Agent is ready to use! It is good practice to call `agent.warm_up()` before running an Agent, which makes sure models are loaded in case that's required."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "2H_0m-Hi0FAU"
   },
   "outputs": [],
   "source": [
    "query = \"What are the latest updates on the Artemis moon mission?\"\n",
    "messages = [ChatMessage.from_user(query)]\n",
    "\n",
    "agent.warm_up()\n",
    "agent_output = agent.run(messages=messages)\n",
    "\n",
    "print(agent_output[\"messages\"][-1].text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0_tne2jaFylV"
   },
   "source": [
    "To render the Agent response in a markdown format, run the code snippet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vfMNiQwjFjOt"
   },
   "outputs": [],
   "source": [
    "from IPython.display import Markdown, display\n",
    "\n",
    "display(Markdown(agent_output[\"messages\"][-1].text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "czMjWwnxPA-3"
   },
   "source": [
    "Let's break down this last example in the tutorial.\n",
    "The **Agent** is the main component that orchestrates the interaction between the LLM and tools.\n",
    "We use **ComponentTool** as a wrapper that allows Haystack components to be used as tools by the agent.\n",
    "The **SuperComponent** wraps entire pipelines so that they can be used as components and thus also as tools.\n",
    "\n",
    "We created a sophisticated search pipeline that:\n",
    "1. Searches the web using SerperDevWebSearch\n",
    "2. Fetches content from the found links\n",
    "3. Converts HTML content to Documents\n",
    "4. Formats the results for the Agent\n",
    "\n",
    "The Agent then uses this pipeline as a tool to gather information and generate comprehensive answers.\n",
    "\n",
    "By the way, did you know that the Agent is a Haystack component itself? That means you can use and combine an Agent in your pipelines just like any other component!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9y4iJE_SrS4K"
   },
   "source": [
    "## What's next\n",
    "\n",
    "🎉 Congratulations! You've learned how to create a tool-calling Agent with Haystack. You can now:\n",
    "- Create simple agents with basic tools\n",
    "- Build complex pipelines with multiple components\n",
    "- Use the Agent component to create sophisticated AI assistants\n",
    "- Combine web search, content fetching, and document processing in your applications\n",
    "\n",
    "If you liked this tutorial, you may also enjoy reusing pipelines from the following examples and make them tools of a powerful Agent:\n",
    "- [Build a GitHub Issue Resolver Agent](https://haystack.deepset.ai/cookbook/github_issue_resolver_agent)\n",
    "- [Building Fallbacks with Conditional Routing](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing)\n",
    "\n",
    "To stay up to date on the latest Haystack developments, you can [subscribe to our newsletter](https://landing.deepset.ai/haystack-community-updates) and [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n",
    "\n",
    "Thanks for reading!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "51Tu2p2C_ZxL"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}