ResearchLink_Recommendation_Engine.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNmZGNXVdzQ6VzZmXhAnrqH",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/beloveddie/AI-Craft/blob/main/ResearchLink_Recommendation_Engine.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ResearchLink Recommendation using Comet's Opik for Tracking, Evaluation, and More\n",
        "\n",
        "This notebook simply aims to present a project that inculcates the value of LLM Evaluation using Opik seamlessly.\n",
        "\n",
        "By leveraging this tool from Comet we are able to track and see into the process flow of our LLM application.\n",
        "\n",
        "Not just that, we can seamlessly compare the output of a given function, prompt or prompt template to others that are different for performance check and just to see which is outperforming which in regards to our set out metrics or end goal."
      ],
      "metadata": {
        "id": "uj2p8uqKA9O5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Installation of required libraries\n",
        "\n",
        "First off, we'll install the necessary libraries for this application"
      ],
      "metadata": {
        "id": "hxmFU_7dCHIg"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "collapsed": true,
        "id": "EnSvCXzSZao2",
        "outputId": "b159a64b-f5e3-42d5-eb58-61e287ee7950"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m285.9/285.9 kB\u001b[0m \u001b[31m7.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 MB\u001b[0m \u001b[31m62.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m232.6/232.6 kB\u001b[0m \u001b[31m17.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m18.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.5/27.5 MB\u001b[0m \u001b[31m56.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.4/76.4 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m162.6/162.6 kB\u001b[0m \u001b[31m12.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m390.3/390.3 kB\u001b[0m \u001b[31m23.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m52.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m70.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install -qU opik litellm requests PyPDF2 sentence-transformers faiss-cpu"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Importation of required libraries"
      ],
      "metadata": {
        "id": "2fE3lDE_CYTg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import opik\n",
        "from litellm import completion\n",
        "import requests\n",
        "import faiss\n",
        "import numpy as np\n",
        "from PyPDF2 import PdfReader\n",
        "from sentence_transformers import SentenceTransformer\n",
        "import xml.etree.ElementTree as ET\n",
        "from google.colab import userdata\n",
        "import os"
      ],
      "metadata": {
        "id": "I3a1VUwPZ43l"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can go ahead and configure and set up the required variables and initialize the necessary `response` object.\n",
        "\n",
        "You see, Opik is such an amazing tool, I'm sincerely loving it. It provides a package called `litellm`. `litellm` abstracts a way some complexities from us and gives us a standard interface to communicate with various LLM providers.\n",
        "\n",
        "Amazing, right? 🤠\n",
        "\n",
        "For this I'll be using my very favourite and go-to LLM provider - you should know by now... COHERE!\n",
        "\n",
        "Cohere's models excel excellently at every task -from reasoning, to generation, to problem solving, to contextual tasks and the likes...\n",
        "\n",
        "Let's go ahead, before I sell Cohere to you. Would be great though."
      ],
      "metadata": {
        "id": "3PN_1ZF6C_ko"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Configure Opik and Cohere for completion using litellm\n",
        "opik.configure(api_key=userdata.get(\"COMET_API_KEY\"))  # Replace with your Opik API key\n",
        "\n",
        "## set ENV variables\n",
        "os.environ[\"COHERE_API_KEY\"] = userdata.get(\"COHERE_API_KEY\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WNrokMw9Z9-0",
        "outputId": "539a3432-6c60-463a-f83b-46101f2d94ca"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "OPIK: Opik is already configured. You can check the settings by viewing the config file at /root/.opik.config\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can access any LLMs from any providers of our choice by a simple and standard interface like so:"
      ],
      "metadata": {
        "id": "3DxG14UuEk5U"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# cohere call\n",
        "response = completion(\n",
        "    model=\"command-r\",\n",
        "    messages = [{ \"content\": \"Hello, how are you?\",\"role\": \"user\"}]\n",
        ")"
      ],
      "metadata": {
        "id": "GBsqDdC8wP2R"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's test this out 😜"
      ],
      "metadata": {
        "id": "JY4rpHXqE0vM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(response.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xzNp6SJ0zhHS",
        "outputId": "4b8fb6bf-c380-4c33-fca3-a1c209f60206"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "I'm doing well, thank you! I am an AI assistant chatbot, so I don't have emotions or feelings, but I am designed to be helpful and provide thorough responses to your queries. Is there anything I can help you with today?\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let's define all our functions for this application.\n",
        "\n",
        "We'll be decorating all of them with the `opik.track` decorator to make such we \"track\" the functions and their outputs respectively on our Comet's Opik Dashboard.\n",
        "\n",
        "AHH, 😫 this is so sleek... Come on, think of \"Explainable AI\", \"Presentable AI\".\n",
        "\n",
        "OPIK!! 😄😃 - a technology that gets me pumped up 😁💯"
      ],
      "metadata": {
        "id": "wTUaa0_ME9bE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def extract_text_from_pdf(pdf_path):\n",
        "    \"\"\"\n",
        "    Extract text from a PDF file, and also extract the title of the paper.\n",
        "    \"\"\"\n",
        "    pdf_reader = PdfReader(pdf_path)\n",
        "\n",
        "    # Get the first page to extract the title (this can vary depending on PDF structure)\n",
        "    first_page = pdf_reader.pages[0]\n",
        "    text = first_page.extract_text()\n",
        "\n",
        "    # Assuming the first line is the title of the paper (this can be customized based on document structure)\n",
        "    title = text.split(\"\\n\")[0]  # Extract the first line as the title (adjust if needed)\n",
        "\n",
        "    # Extract the rest of the text (excluding title)\n",
        "    body_text = text[len(title):]\n",
        "\n",
        "    # Return both the title and the body text\n",
        "    return title, body_text"
      ],
      "metadata": {
        "id": "E4atzmmE8egw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def summarize_current_summary(document, instruction, current_summary, model=\"command-r\"):\n",
        "    \"\"\"\n",
        "    Generate a summary using Comet's command-r model with the Standardized litellm interface.\n",
        "    \"\"\"\n",
        "    prompt = f\"\"\"\n",
        "    Document: {document}\n",
        "    Current summary: {current_summary}\n",
        "    Instruction: {instruction}\n",
        "\n",
        "    Revise the summary to make it more concise, technically accurate, and entity-dense, while aligning with the given instruction.\n",
        "\n",
        "    \"\"\"\n",
        "\n",
        "    # Generate completion using command-r\n",
        "    response = completion(\n",
        "        model=model,  # Use command-r\n",
        "        messages = [{ \"content\": prompt,\"role\": \"user\"}],\n",
        "        max_token=1024,\n",
        "    )\n",
        "\n",
        "    return response.choices[0].message.content  # Extract the summary from the response"
      ],
      "metadata": {
        "id": "5gZXtLVfpuNV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def fetch_arxiv_papers(query, max_results=10):\n",
        "    \"\"\"\n",
        "    Fetch research papers from ArXiv based on a search query.\n",
        "    \"\"\"\n",
        "    base_url = \"http://export.arxiv.org/api/query\"\n",
        "    params = {\n",
        "        \"search_query\": query,\n",
        "        \"start\": 0,\n",
        "        \"max_results\": max_results,\n",
        "        \"sortBy\": \"relevance\",\n",
        "        \"sortOrder\": \"descending\",\n",
        "    }\n",
        "\n",
        "    response = requests.get(base_url, params=params)\n",
        "    if response.status_code == 200:\n",
        "        root = ET.fromstring(response.content)\n",
        "        papers = []\n",
        "        for entry in root.findall(\"{http://www.w3.org/2005/Atom}entry\"):\n",
        "            paper = {\n",
        "                \"title\": entry.find(\"{http://www.w3.org/2005/Atom}title\").text,\n",
        "                \"summary\": entry.find(\"{http://www.w3.org/2005/Atom}summary\").text,\n",
        "                \"authors\": [author.find(\"{http://www.w3.org/2005/Atom}name\").text for author in entry.findall(\"{http://www.w3.org/2005/Atom}author\")],\n",
        "                \"link\": entry.find(\"{http://www.w3.org/2005/Atom}id\").text,\n",
        "            }\n",
        "            papers.append(paper)\n",
        "        return papers\n",
        "    else:\n",
        "        raise Exception(f\"Failed to fetch data from ArXiv API: {response.status_code}\")"
      ],
      "metadata": {
        "id": "wQ_nozkhbRoQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def generate_embedding(summary):\n",
        "    embedding_model = SentenceTransformer('all-MiniLM-L6-v2')\n",
        "    return embedding_model.encode(summary)"
      ],
      "metadata": {
        "id": "ytlpLGwbbgli"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def add_to_index(title, summary, embedding, faiss_index, external_papers):\n",
        "    faiss_index.add(np.array([embedding]))\n",
        "    external_papers.append({\"title\": title, \"summary\": summary, \"embedding\": embedding})"
      ],
      "metadata": {
        "id": "KiMaQUEPblh7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track\n",
        "def recommend_with_external_and_uploaded(query_embedding, faiss_index, external_papers, top_k=5):\n",
        "    distances, indices = faiss_index.search(np.array([query_embedding]), k=top_k)\n",
        "    recommendations = [{\"title\": external_papers[i]['title'], \"summary\": external_papers[i]['summary'] } for i in indices[0]]\n",
        "    return recommendations"
      ],
      "metadata": {
        "id": "VjHd2NAIbp9v"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@opik.track(project_name=\"ResearchLink Recommendation Engine\")\n",
        "def full_pipeline(pdf_path, query, top_k=5):\n",
        "    \"\"\"\n",
        "    Integrates summarization, external data fetching, and recommendations with Anthropic's Claude.\n",
        "    \"\"\"\n",
        "    # Step 1: Summarize uploaded PDF using Claude\n",
        "    title, document = extract_text_from_pdf(pdf_path)\n",
        "    # print(document)\n",
        "    instruction = \"Summarize the key findings and contributions of this paper.\"\n",
        "    summary = summarize_current_summary(document=document, instruction=instruction, current_summary=\"\")\n",
        "    embedding = generate_embedding(summary)\n",
        "\n",
        "    # Initialize FAISS index\n",
        "    embedding_dim = 384  # Dimension of the embedding model\n",
        "    faiss_index = faiss.IndexFlatL2(embedding_dim)\n",
        "    external_papers = []\n",
        "\n",
        "    # Step 2: Add the summary to the FAISS index\n",
        "    add_to_index(title, summary, embedding, faiss_index, external_papers)\n",
        "\n",
        "    # Step 3: Fetch related papers from ArXiv\n",
        "    fetched_papers = fetch_arxiv_papers(query, max_results=10)\n",
        "    for paper in fetched_papers:\n",
        "        paper_embedding = generate_embedding(paper['summary'])\n",
        "        add_to_index(paper['title'], paper['summary'], paper_embedding, faiss_index, external_papers)\n",
        "\n",
        "    # Step 4: Generate recommendations\n",
        "    recommendations = recommend_with_external_and_uploaded(embedding, faiss_index, external_papers, top_k=top_k)\n",
        "\n",
        "    return title, summary, recommendations"
      ],
      "metadata": {
        "id": "0IbbUo4xiV_M"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, let's enable paper upload from our Colab notebook... Cool? 😎"
      ],
      "metadata": {
        "id": "htK5LxPeGpsc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 131
        },
        "id": "Df__tEG_b3sQ",
        "outputId": "5e73d9db-7c34-401f-85b2-7c1d63b06698"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-d478459f-c6f3-400f-91bc-73ccd594261d\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-d478459f-c6f3-400f-91bc-73ccd594261d\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving 1706.03762v7.pdf to 1706.03762v7.pdf\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pdf_path = list(uploaded.keys())[0] # Get the first uploaded file's path\n",
        "query = \"LLM Agents and Transformer Architectures\" # Example search query for related papers"
      ],
      "metadata": {
        "id": "WA12IW79evA_"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let's get the `title`, `summary`, and `recommendations` from our `full_pipeline` function."
      ],
      "metadata": {
        "id": "SLUflWv0G3AP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "title, summary, recommendations = full_pipeline(pdf_path, query)"
      ],
      "metadata": {
        "id": "UHRgSHL9e0HB",
        "collapsed": true
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Display the summary and recommendations\n",
        "print(f\"Summary of Uploaded Paper [{title}]:\")\n",
        "print(summary)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "X07wqQZMq2eF",
        "outputId": "adc73e46-b1e5-4755-8fc3-e8a7740fe1a5"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Summary of Uploaded Paper [Provided proper attribution is provided, Google hereby grants permission to]:\n",
            "## Summary:\n",
            "\n",
            "This paper introduces a novel neural network architecture, the Transformer, designed for sequence transduction tasks. The Transformer simplifies existing models by relying solely on attention mechanisms, eliminating the need for recurrent or convolutional layers. The authors demonstrate its effectiveness in machine translation, achieving state-of-the-art performance on English-to-German and English-to-French translation tasks. The Transformer outperforms previous models, improving BLEU scores by a significant margin and reducing training time and costs. Additionally, the model's versatility is showcased through successful application in English constituency parsing with varying training data sizes.\n",
            "\n",
            "## Revised Summary:\n",
            "\n",
            "The Transformer architecture, a new attention-based model, outperforms traditional sequence transduction models. It achieves 28.4 BLEU on WMT 2014 English-to-German and 41.8 BLEU on English-to-French translation tasks, surpassing existing methods. The Transformer's efficiency is evident in faster training and lower costs, making it a significant contribution to neural machine translation.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's check out the recommendations we've got."
      ],
      "metadata": {
        "id": "BR6FZeYlL67L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "print(\"\\nRecommended Papers:\")\n",
        "for rec in recommendations:\n",
        "    print(f\"Title: {rec['title']}\")"
      ],
      "metadata": {
        "id": "a25pvv7iqANu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "6cefd6f7-62f1-427c-c179-f9f316b9f0d1"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Recommended Papers:\n",
            "Title: Provided proper attribution is provided, Google hereby grants permission to\n",
            "Title: Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large\n",
            "  Language Models\n",
            "Title: Advancing Transformer Architecture in Long-Context Large Language\n",
            "  Models: A Comprehensive Survey\n",
            "Title: Towards Collaborative Intelligence: Propagating Intentions and Reasoning\n",
            "  for Multi-Agent Coordination with Large Language Models\n",
            "Title: Targeting the Core: A Simple and Effective Method to Attack RAG-based\n",
            "  Agents via Direct LLM Manipulation\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(recommendations)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "WheR1WNb3MIR",
        "outputId": "58c87877-7016-4758-9428-a5e7372223bd"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "recommendations"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "aT7CgTLV3Yf9",
        "outputId": "f4f1ff76-b177-4958-adc1-d2a7b2581423"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'title': 'Provided proper attribution is provided, Google hereby grants permission to',\n",
              "  'summary': \"## Summary:\\n\\nThis paper introduces a novel neural network architecture, the Transformer, designed for sequence transduction tasks. The Transformer simplifies existing models by relying solely on attention mechanisms, eliminating the need for recurrent or convolutional layers. The authors demonstrate its effectiveness in machine translation, achieving state-of-the-art performance on English-to-German and English-to-French translation tasks. The Transformer outperforms previous models, improving BLEU scores by a significant margin and reducing training time and costs. Additionally, the model's versatility is showcased through successful application in English constituency parsing with varying training data sizes.\\n\\n## Revised Summary:\\n\\nThe Transformer architecture, a new attention-based model, outperforms traditional sequence transduction models. It achieves 28.4 BLEU on WMT 2014 English-to-German and 41.8 BLEU on English-to-French translation tasks, surpassing existing methods. The Transformer's efficiency is evident in faster training and lower costs, making it a significant contribution to neural machine translation.\"},\n",
              " {'title': 'Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large\\n  Language Models',\n",
              "  'summary': '  Simultaneous Machine Translation (SiMT) generates target translations while\\nreading the source sentence. It relies on a policy to determine the optimal\\ntiming for reading sentences and generating translations. Existing SiMT methods\\ngenerally adopt the traditional Transformer architecture, which concurrently\\ndetermines the policy and generates translations. While they excel at\\ndetermining policies, their translation performance is suboptimal. Conversely,\\nLarge Language Models (LLMs), trained on extensive corpora, possess superior\\ngeneration capabilities, but it is difficult for them to acquire translation\\npolicy through the training methods of SiMT. Therefore, we introduce\\nAgent-SiMT, a framework combining the strengths of LLMs and traditional SiMT\\nmethods. Agent-SiMT contains the policy-decision agent and the translation\\nagent. The policy-decision agent is managed by a SiMT model, which determines\\nthe translation policy using partial source sentence and translation. The\\ntranslation agent, leveraging an LLM, generates translation based on the\\npartial source sentence. The two agents collaborate to accomplish SiMT.\\nExperiments demonstrate that Agent-SiMT attains state-of-the-art performance.\\n'},\n",
              " {'title': 'Advancing Transformer Architecture in Long-Context Large Language\\n  Models: A Comprehensive Survey',\n",
              "  'summary': '  Transformer-based Large Language Models (LLMs) have been applied in diverse\\nareas such as knowledge bases, human interfaces, and dynamic agents, and\\nmarking a stride towards achieving Artificial General Intelligence (AGI).\\nHowever, current LLMs are predominantly pretrained on short text snippets,\\nwhich compromises their effectiveness in processing the long-context prompts\\nthat are frequently encountered in practical scenarios. This article offers a\\ncomprehensive survey of the recent advancement in Transformer-based LLM\\narchitectures aimed at enhancing the long-context capabilities of LLMs\\nthroughout the entire model lifecycle, from pre-training through to inference.\\nWe first delineate and analyze the problems of handling long-context input and\\noutput with the current Transformer-based models. We then provide a taxonomy\\nand the landscape of upgrades on Transformer architecture to solve these\\nproblems. Afterwards, we provide an investigation on wildly used evaluation\\nnecessities tailored for long-context LLMs, including datasets, metrics, and\\nbaseline models, as well as optimization toolkits such as libraries,\\nframeworks, and compilers to boost the efficacy of LLMs across different stages\\nin runtime. Finally, we discuss the challenges and potential avenues for future\\nresearch. A curated repository of relevant literature, continuously updated, is\\navailable at https://github.com/Strivin0311/long-llms-learning.\\n'},\n",
              " {'title': 'Towards Collaborative Intelligence: Propagating Intentions and Reasoning\\n  for Multi-Agent Coordination with Large Language Models',\n",
              "  'summary': '  Effective collaboration in multi-agent systems requires communicating goals\\nand intentions between agents. Current agent frameworks often suffer from\\ndependencies on single-agent execution and lack robust inter-module\\ncommunication, frequently leading to suboptimal multi-agent reinforcement\\nlearning (MARL) policies and inadequate task coordination. To address these\\nchallenges, we present a framework for training large language models (LLMs) as\\ncollaborative agents to enable coordinated behaviors in cooperative MARL. Each\\nagent maintains a private intention consisting of its current goal and\\nassociated sub-tasks. Agents broadcast their intentions periodically, allowing\\nother agents to infer coordination tasks. A propagation network transforms\\nbroadcast intentions into teammate-specific communication messages, sharing\\nrelevant goals with designated teammates. The architecture of our framework is\\nstructured into planning, grounding, and execution modules. During execution,\\nmultiple agents interact in a downstream environment and communicate intentions\\nto enable coordinated behaviors. The grounding module dynamically adapts\\ncomprehension strategies based on emerging coordination patterns, while\\nfeedback from execution agents influnces the planning module, enabling the\\ndynamic re-planning of sub-tasks. Results in collaborative environment\\nsimulation demonstrate intention propagation reduces miscoordination errors by\\naligning sub-task dependencies between agents. Agents learn when to communicate\\nintentions and which teammates require task details, resulting in emergent\\ncoordinated behaviors. This demonstrates the efficacy of intention sharing for\\ncooperative multi-agent RL based on LLMs.\\n'},\n",
              " {'title': 'Targeting the Core: A Simple and Effective Method to Attack RAG-based\\n  Agents via Direct LLM Manipulation',\n",
              "  'summary': '  AI agents, powered by large language models (LLMs), have transformed\\nhuman-computer interactions by enabling seamless, natural, and context-aware\\ncommunication. While these advancements offer immense utility, they also\\ninherit and amplify inherent safety risks such as bias, fairness,\\nhallucinations, privacy breaches, and a lack of transparency. This paper\\ninvestigates a critical vulnerability: adversarial attacks targeting the LLM\\ncore within AI agents. Specifically, we test the hypothesis that a deceptively\\nsimple adversarial prefix, such as \\\\textit{Ignore the document}, can compel\\nLLMs to produce dangerous or unintended outputs by bypassing their contextual\\nsafeguards. Through experimentation, we demonstrate a high attack success rate\\n(ASR), revealing the fragility of existing LLM defenses. These findings\\nemphasize the urgent need for robust, multi-layered security measures tailored\\nto mitigate vulnerabilities at the LLM level and within broader agent-based\\narchitectures.\\n'}]"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "YOHH!!!\n",
        "\n",
        "And we've got all these tracked and ready for easy and seamless visualization, evaluation and monitoring on the Comet's Opik platform.\n",
        "\n",
        "The value of Opik is tremendous, I've experimented with various experiment trackers and the likes... Opik from Comet gave me exactly what I envisioned deep down and even more.\n",
        "\n",
        "I'm still getting to know and apply this amazing tool to my ML/LLM workflow. You could do same too 😉 😁\n",
        "\n",
        "Happy and joyful Cheers, my friend.\n",
        "\n",
        "Eddie Otudor"
      ],
      "metadata": {
        "id": "LeL-dboZHM7e"
      }
    }
  ]
}