-It works! Let's go on with the conversation.
-```python PYTHON
+
+
+
-
+
-
-More broadly, the implementation of a RAG pipeline - including parsing and chunking of documents, along with the embedding and retrieval of the chunks - is critical to the accuracy of grounded answers. Additionally, it is sometimes not sufficient to merely retrieve the answers; a user may want further postprocessing performed on the output. This use case would benefit from giving the model access to tools.
-## Objective
+## Motivation
+Asking questions over documents continues to be an important retrieval augmented generation (RAG) task. However, the document complexity can significantly influence overall RAG performance, particularly when the documents are PDFs that contain a mix of text and tables. Finding an optimal strategy to parse this information, chunk, embed and retrieve it is thus quite critical to obtaining accurate results. Furthermore, if the questions being asked over the retrieved documents require mathematical reasoning, then having a model that can validate those operations is quite useful.
-In this notebook, we will guide you through best practices for setting up a RAG pipeline to process documents that contain both tables and text. We will also demonstrate how to create a [ReAct](https://python.langchain.com/v0.1/docs/modules/agents/agent_types/react/) agent with a Cohere model, and then give the agent access to a RAG pipeline tool to improve accuracy. The general structure of the notebook is as follows:
+## Objective
+In this notebook we will guide you through the best practices of setting up a RAG pipeline to process documents that contain both tables and text. In addition, we will show you how to create a Cohere ReAct agent with access to a RAG pipeline tool to improve accuracy. The general structure of the nb is as follows:
-- individual components around parsing, retrieval and generation are covered for documents with mixed tabular and textual data
-- a class object is created that can be used to instantiate the pipeline with parametric input
-- the RAG pipeline is then used as a tool for a Cohere ReACT agent
+1. individual components around parsing, retrieval and generation are covered for documents with mixed tabular and textual data
+2. a class object is created that can be used to instantiate the pipeline with parametric input
+3. the RAG pipeline is then used as a tool for a cohere react agent
# Reference Documents
+we recommend the following as a guide on doing [semi-structured RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb)
-We recommend the following notebook as a guide to [semi-structured RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb).
+we recommend this notebook to explore various parsing techniques for [PDFs](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Document_Parsing_For_Enterprises.ipynb)
-We also recommend the following notebook to explore various parsing techniques for [PDFs](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Document_Parsing_For_Enterprises.ipynb).
+various langchain supported [parsers](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf/)
-Various LangChain-supported parsers can be found [here](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf/).
+## Table of Contents
+- Section 1
+ - [Parsing](#sec_step1)
+ - [Vector Store Setup](#sec_step2)
+ - [RAG Pipeline](#sec_step3)
+- Section 2
+ - [RAG Pipeline Class](#sec_step4)
+- Section 3
+ - [ReAct Agent with RAG Tool](#sec_step5)
## Install Dependencies
-```python PYTHON
+
+```python
# there may be other dependencies that will need installation
# ! pip install --quiet langchain langchain_cohere langchain_experimental
# !pip --quiet install faiss-cpu tiktoken
@@ -56,7 +66,8 @@ Various LangChain-supported parsers can be found [here](https://python.langchain
# !pip install chromadb
```
-```python PYTHON
+
+```python
# LLM
import os
from langchain.text_splitter import RecursiveCharacterTextSplitter
@@ -78,23 +89,26 @@ import pandas as pd
from datasets import load_dataset
from joblib import Parallel, delayed
+
os.environ['COHERE_API_KEY'] = ""
```
-# Parsing [#sec_step1]
+
+# Parsing
To improve RAG performance on PDFs with mixed types (text and tables), we investigated a number of parsing and chunking strategies from various libraries:
+- PyPDFLoader (LC)
+- LlamaParse (Llama-Index)
+- Unstructured IO
-- [PyPDFLoader (LC)](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html)
-- [LlamaParse](https://docs.llamaindex.ai/en/stable/llama_cloud/llama_parse/) (Llama-Index)
-- [Unstructured](https://unstructured.io/)
We have found that the best option for parsing is unstructured.io since the parser can:
-
- separate tables from text
- automatically chunk the tables and text by title during the parsing step so that similar elements are grouped
-```python PYTHON
+
+
+```python
# UNSTRUCTURED pdf loader
# Get elements
raw_pdf_elements = partition_pdf(
@@ -117,7 +131,14 @@ raw_pdf_elements = partition_pdf(
```
-```python PYTHON
+ This function will be deprecated in a future release and `unstructured` will simply use the DEFAULT_MODEL from `unstructured_inference.model.base` to set default model name
+ Some weights of the model checkpoint at microsoft/table-transformer-structure-recognition were not used when initializing TableTransformerForObjectDetection: ['model.backbone.conv_encoder.model.layer4.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer3.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer2.0.downsample.1.num_batches_tracked']
+ - This IS expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+ - This IS NOT expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+
+
+
+```python
# extract table and textual objects from parser
class Element(BaseModel):
type: str
@@ -140,25 +161,27 @@ text_elements = [e for e in categorized_elements if e.type == "text"]
print(len(text_elements))
```
-```txt title="Output"
-14
-24
-```
+ 14
+ 24
-# Vector Store Setup [#sec_step2]
-There are many options for setting up a vector store. Here, we show how to do so using [Chroma](https://www.trychroma.com/) and Langchain's Multi-vector retrieval.
-As the name implies, multi-vector retrieval allows us to store multiple vectors per document; for instance, for a single document chunk, one could keep embeddings for both the chunk itself, and a summary of that document. A summary may be able to distill more accurately what a chunk is about, leading to better retrieval.
+
+# Vector Store Setup
-You can read more about this here: https://python.langchain.com/docs/modules/data_connection/retrievers/multi_vector/
+There are many options to setup a vector store. Here we show how to set one up using Chroma and Langchains Multi-vector retrieval.
+We use multi-vector retrieval because oftentimes a summary may be able to distill more accurately what a chunk is about, leading to better retrieval.
-Below, we demonstrate the following process:
+You can read more about this here: https://python.langchain.com/docs/modules/data_connection/retrievers/multi_vector/
+The process is as follows:
- summaries of each chunk are embedded
- during inference, the multi-vector retrieval returns the full context document related to the summary
-```python PYTHON
+
+```python
+
co = cohere.Client()
+
def get_chat_output(message, preamble, chat_history, model, temp, documents=None):
return co.chat(
message=message,
@@ -187,9 +210,11 @@ def rerank_cohere(query, returned_documents,model:str="rerank-multilingual-v3.0"
```
-```python PYTHON
+
+```python
# generate table and text summaries
-prompt_text = """You are an assistant tasked with summarizing tables and text. \
+
+prompt_text = """You are an assistant tasked with summarizing tables and text. \
Give a concise summary of the table or text. Table or text chunk: {element}. Only provide the summary and no other text."""
table_prompts = [prompt_text.format(element=i.text) for i in table_elements]
@@ -200,7 +225,8 @@ tables = [i.text for i in table_elements]
texts = [i.text for i in text_elements]
```
-```python PYTHON
+
+```python
# The vectorstore to use to index the child chunks
vectorstore = Chroma(collection_name="summaries", embedding_function=CohereEmbeddings())
# The storage layer for the parent documents
@@ -230,15 +256,17 @@ retriever.vectorstore.add_documents(summary_tables)
retriever.docstore.mset(list(zip(table_ids, tables)))
```
-# RAG Pipeline [#sec_step3]
+
+# RAG Pipeline
-With our database in place, we can run queries against it. The query process can be broken down into the following steps:
+the query process can be broken down into the following steps:
-- augment the query, this really helps retrieve all the relevant information
-- use each augmented query to retrieve the top k docs and then rerank them
-- concatenate all the shortlisted/reranked docs and pass them to the generation model
+1. augment the query, this really helps retrieve all the relevant information
+2. use each augmented query to retrieve the top k docs and then rerank them
+3. concatenate all the shortlisted/reranked docs and pass them to the generation model
-```python PYTHON
+
+```python
def process_query(query, retriever):
"""Runs query augmentation, retrieval, rerank and final generation in one call."""
augmented_queries=co.chat(message=query,model='command-r-plus',temperature=0.2, search_queries_only=True)
@@ -253,9 +281,9 @@ def process_query(query, retriever):
else:
#no queries will be run through RAG
documents = None
-
+
preamble = """
-## Task & Context
+## Task & Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
@@ -264,8 +292,8 @@ Unless the user asks for a different style of answer, you should answer in full
model = 'command-r-plus'
temp = 0.2
-
-
+
+
response = co.chat(
message=query,
documents=documents,
@@ -275,18 +303,17 @@ Unless the user asks for a different style of answer, you should answer in full
)
final_answer_docs="""The final answer is from the documents below:
-
+
{docs}""".format(docs=str(response.documents))
final_answer = response.text
return final_answer, final_answer_docs
```
-## Example
+## Example
-We can now test out a query. In this example, the final answer can be found on page 12 of the PDF, which aligns with the response provided by the model:
-```python PYTHON
+```python
query = "what are the charges for services in 2022"
final_answer, final_answer_docs = process_query(query, retriever)
print(final_answer)
@@ -294,24 +321,27 @@ print(final_answer_docs)
chat_history=[{'role':"USER", 'message':query},{'role':"CHATBOT", 'message':f'The final answer is: {final_answer}.' + final_answer_docs}]
+
```
-```txt title="Output"
-The charges for services in 2022 were $5,266 million.
-The final answer is from the documents below:
+ The charges for services in 2022 were $5,266 million.
+ The final answer is from the documents below:
+
+ [{'id': 'doc_0', 'snippet': 'Program and General Revenues FY 2023 FY 2022 FY 2021 Category (in millions) Charges for Services (CS) $5,769 $5,266 $5,669 Operating Grants and Contributions (OGC) 27,935 31,757 28,109 Capital Grants and Contributions (CGC) 657 656 675 Real Estate Taxes (RET) 31,502 29,507 31,421 Sales and Use Taxes (SUT) 10,577 10,106 7,614 Personal Income Taxes (PIT) 15,313 15,520 15,795 Income Taxes, Other (ITO) 13,181 9,521 9,499 Other Taxes* (OT) 3,680 3,777 2,755 Investment Income* (II) 694 151 226 Unrestricted Federal and State Aid (UFSA) 234 549 108 Other* (O) Total Program and General Revenues - Primary Government 2,305 $110,250 $107,535 $104,176 708 725', 'title': 'chunk 0'}]
- [{'id': 'doc_0', 'snippet': 'Program and General Revenues FY 2023 FY 2022 FY 2021 Category (in millions) Charges for Services (CS) $5,769 $5,266 $5,669 Operating Grants and Contributions (OGC) 27,935 31,757 28,109 Capital Grants and Contributions (CGC) 657 656 675 Real Estate Taxes (RET) 31,502 29,507 31,421 Sales and Use Taxes (SUT) 10,577 10,106 7,614 Personal Income Taxes (PIT) 15,313 15,520 15,795 Income Taxes, Other (ITO) 13,181 9,521 9,499 Other Taxes* (OT) 3,680 3,777 2,755 Investment Income* (II) 694 151 226 Unrestricted Federal and State Aid (UFSA) 234 549 108 Other* (O) Total Program and General Revenues - Primary Government 2,305 $110,250 $107,535 $104,176 708 725', 'title': 'chunk 0'}]
-```
-### Chat History Management
+the final answer is correct based on page 12 in the PDF and we can see that the information retrieved is linked to that table :)
+
+### chat history management
-In the example below, we ask a follow up question that relies on the chat history, but does not require a rerun of the RAG pipeline.
+an example of asking a follow up question that relies on the chat history but does not require a re-run of RAG.
-We detect questions that do not require RAG by examining the `search_queries` object returned by calling `co.chat` to generate candidate queries to answer our question. If this object is empty, then the model has determined that a document query is not needed to answer the question.
+The search_queries_only flag can be used to determine whether RAG needs to be rerun or not i.e. it can help easily identify if the query passed needs retrieval.
-In the example below, the `else` statement is invoked based on `query2`. We still pass in the chat history, allowing the question to be answered with only the prior context.
+In the example below, the else statement is invoked based on query2. In the else we pass in history without documents as the new query does not need to call the RAG pipeline
-```python PYTHON
+
+```python
query2='divide this by two'
augmented_queries=co.chat(message=query2,model='command-r-plus',temperature=0.2, search_queries_only=True)
if augmented_queries.search_queries:
@@ -326,32 +356,34 @@ else:
chat_history=chat_history,
temperature=0.3
)
-
+
print("Final answer:")
print(response.text)
```
-```txt title="Output"
RAG is not needed
Final answer:
The result of dividing the charges for services in 2022 by two is $2,633.
-```
----
-# RAG Pipeline Class [#sec_step4]
+## ------------------------------------------------------- ##
+
+
+# RAG Pipeline Class
+
+Here we connect all the pieces discussed above into one class object that is then used as a tool for a cohere react agent. This class is to help consolidate and clarify the key parameters used to define the RAG pipeline.
-Here, we connect all of the pieces discussed above into one class object, which is then used as a tool for a Cohere ReAct agent. This class definition consolidates and clarify the key parameters used to define the RAG pipeline.
-```python PYTHON
+```python
co = cohere.Client()
```
-```python PYTHON
+
+```python
class Element(BaseModel):
type: str
text: Any
-
+
class RAG_pipeline():
def __init__(self,paths):
self.embedding_model="embed-english-v3.0"
@@ -362,12 +394,12 @@ class RAG_pipeline():
self.top_k_rerank=3
self.temperature=0.2
self.preamble="""
-## Task & Context
+## Task & Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.
-"""
+"""
self.n_jobs=10 #number of parallel processes to run summarization of chunks
self.extract_images_in_pdf=False
self.infer_table_structure=True
@@ -378,7 +410,7 @@ Unless the user asks for a different style of answer, you should answer in full
self.image_output_dir_path='.'
self.paths = paths
self.parse_and_build_retriever()
-
+
def parse_and_build_retriever(self,):
#step1, parse pdfs
# if condition just for debugging since perf_audit.pdf is parsed in the prev step, no need to rerun
@@ -393,7 +425,7 @@ Unless the user asks for a different style of answer, you should answer in full
self.text_summaries=text_summaries
#setup the multivector retriever
self.make_retriever(tables, table_summaries, texts, text_summaries)
-
+
def extract_text_and_tables(self,parsed_pdf_list):
# extract table and textual objects from parser
# Categorize by type
@@ -406,11 +438,11 @@ Unless the user asks for a different style of answer, you should answer in full
categorized_elements.append(Element(type="table", text=str(element)))
elif "unstructured.documents.elements.CompositeElement" in str(type(element)):
categorized_elements.append(Element(type="text", text=str(element)))
-
+
# Tables
table_elements = [e for e in categorized_elements if e.type == "table"]
print(len(table_elements))
-
+
# Text
text_elements = [e for e in categorized_elements if e.type == "text"]
print(len(text_elements))
@@ -418,7 +450,7 @@ Unless the user asks for a different style of answer, you should answer in full
all_text_elements.extend(text_elements)
return all_table_elements, all_text_elements
-
+
def parse_pdfs(self, paths):
path_raw_elements = []
@@ -443,12 +475,12 @@ Unless the user asks for a different style of answer, you should answer in full
path_raw_elements.append(raw_pdf_elements)
print('PDFs parsed')
return path_raw_elements
-
+
def get_chat_output(self,message, preamble, model, temp):
# print("**message")
# print(message)
-
+
response=co.chat(
message=message,
preamble=preamble,
@@ -463,7 +495,7 @@ Unless the user asks for a different style of answer, you should answer in full
"""Parallel processing of chat endpoint calls."""
responses = Parallel(n_jobs=n_jobs, prefer="threads")(delayed(self.get_chat_output)(prompt,preamble,model,temp) for prompt in prompts)
return responses
-
+
def rerank_cohere(self,query, returned_documents,model, top_n):
response = co.rerank(
query=query,
@@ -478,9 +510,9 @@ Unless the user asks for a different style of answer, you should answer in full
def generate_summaries(self,table_elements,text_elements):
# generate table and text summaries
- summarize_prompt = """You are an assistant tasked with summarizing tables and text. \
+ summarize_prompt = """You are an assistant tasked with summarizing tables and text. \
Give a concise summary of the table or text. Table or text chunk: {element}. Only provide the summary and no other text."""
-
+
table_prompts = [summarize_prompt.format(element=i.text) for i in table_elements]
table_summaries = self.parallel_proc_chat(table_prompts,self.preamble,self.summary_model,self.temperature,self.n_jobs)
text_prompts = [summarize_prompt.format(element=i.text) for i in text_elements]
@@ -521,7 +553,7 @@ Unless the user asks for a different style of answer, you should answer in full
retriever.docstore.mset(list(zip(table_ids, tables)))
self.retriever = retriever
print('retriever built')
-
+
def process_query(self,query):
"""Runs query augmentation, retrieval, rerank and generation in one call."""
augmented_queries=co.chat(message=query,model=self.generation_model,temperature=self.temperature, search_queries_only=True)
@@ -535,7 +567,7 @@ Unless the user asks for a different style of answer, you should answer in full
documents = [{"title": f"chunk {i}", "snippet": reranked_docs[i]} for i in range(len(reranked_docs))]
else:
documents = None
-
+
response = co.chat(
message=query,
documents=documents,
@@ -543,40 +575,38 @@ Unless the user asks for a different style of answer, you should answer in full
model=self.generation_model,
temperature=self.temperature
)
-
+
final_answer_docs="""The final answer is from the documents below:
-
+
{docs}""".format(docs=str(response.documents))
-
+
final_answer = response.text
return final_answer, final_answer_docs
```
-```python PYTHON
+
+```python
rag_object=RAG_pipeline(paths=["city_ny_popular_fin_report.pdf"])
```
This function will be deprecated in a future release and `unstructured` will simply use the DEFAULT_MODEL from `unstructured_inference.model.base` to set default model name
-```txt title="Output"
-PDFs parsed
-14
-24
-summaries generated
-retriever built
-```
-# Cohere ReAct Agent with RAG Tool [#sec_step5]
+ PDFs parsed
+ 14
+ 24
+ summaries generated
+ retriever built
+
-Finally, we build a simple agent that utilizes the RAG pipeline defined above. We do this by granting the agent access to two tools:
+
+# Cohere REACT Agent with RAG Tool
-- the end-to-end RAG pipeline
-- a Python interpreter
+we build a simple agent using the RAG pipeline previously defined. The e2e rag pipeline is provided as a tool in addition to a python tool. The premise in coupling these tools is so that the mathematical steps can be done using a python tool to improve accuracy.
-The intention behind coupling these tools is to enable the model to perform mathematical and other postprocessing operations on RAG outputs using Python.
-```python PYTHON
+```python
from langchain.agents import Tool
from langchain_experimental.utilities import PythonREPL
from langchain.agents import AgentExecutor
@@ -591,20 +621,20 @@ class react_agent():
def __init__(self,rag_retriever,model="command-r-plus",temperature=0.2):
self.llm = ChatCohere(model=model, temperature=temperature)
self.preamble="""
-## Task & Context
+## Task & Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.
## Guidelines
-You are an expert who answers the user's question.
+You are an expert who answers the user's question.
You have access to a vectorsearch tool that will use your query to search through documents and find the relevant answer.
You also have access to a python interpreter tool which you can use to run code for mathematical operations.
"""
self.get_tools(rag_retriever)
self.build_agent()
-
+
def get_tools(self,rag_retriever):
@tool
def vectorsearch(query: str):
@@ -617,7 +647,7 @@ You also have access to a python interpreter tool which you can use to run code
query: str = Field(description="the users query")
vectorsearch.args_schema = vectorsearch_inputs
-
+
python_repl = PythonREPL()
python_tool = Tool(
name="python_repl",
@@ -656,79 +686,90 @@ You also have access to a python interpreter tool which you can use to run code
"preamble": self.preamble,
})
return response
-
+
```
-```python PYTHON
+
+```python
agent_object=react_agent(rag_retriever=rag_object)
```
-```python PYTHON
+
+```python
step1_response=agent_object.run_agent("what are the charges for services in 2022 and 2023")
```
-```txt title="Output"
-[1m> Entering new AgentExecutor chain...[0m
-[32;1m[1;3m
-I will search for the charges for services in 2022 and 2023.
-{'tool_name': 'vectorsearch', 'parameters': {'query': 'charges for services in 2022 and 2023'}}
-[0m[36;1m[1;3mThe charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.The final answer is from the documents below:
+
+
+ [1m> Entering new AgentExecutor chain...[0m
+ [32;1m[1;3m
+ I will search for the charges for services in 2022 and 2023.
+ {'tool_name': 'vectorsearch', 'parameters': {'query': 'charges for services in 2022 and 2023'}}
+ [0m[36;1m[1;3mThe charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.The final answer is from the documents below:
+
+ [{'id': 'doc_0', 'snippet': 'Program and General Revenues FY 2023 FY 2022 FY 2021 Category (in millions) Charges for Services (CS) $5,769 $5,266 $5,669 Operating Grants and Contributions (OGC) 27,935 31,757 28,109 Capital Grants and Contributions (CGC) 657 656 675 Real Estate Taxes (RET) 31,502 29,507 31,421 Sales and Use Taxes (SUT) 10,577 10,106 7,614 Personal Income Taxes (PIT) 15,313 15,520 15,795 Income Taxes, Other (ITO) 13,181 9,521 9,499 Other Taxes* (OT) 3,680 3,777 2,755 Investment Income* (II) 694 151 226 Unrestricted Federal and State Aid (UFSA) 234 549 108 Other* (O) Total Program and General Revenues - Primary Government 2,305 $110,250 $107,535 $104,176 708 725', 'title': 'chunk 0'}][0m[32;1m[1;3mRelevant Documents: 0
+ Cited Documents: 0
+ Answer: The charges for services in 2022 were $5,266 million and in 2023 were $5,769 million.
+ Grounded answer: The charges for services in 
+
Large language models give machines a vastly improved representation and understanding of language. These abilities give developers more options for content recommendation, analysis, and filtering.
In this notebook we take thousands of the most popular posts from Hacker News and demonstrate some of these functionalities:
1. Given an existing post title, retrieve the most similar posts (nearest neighbor search using embeddings)
-2. Given a query that we write, retrieve the most similar posts
-3. Plot the archive of articles by similarity (where similar posts are close together and different ones are far)
-4. Cluster the posts to identify the major common themes
-5. Extract major keywords from each cluster so we can identify what the clsuter is about
-6. (Experimental) Name clusters with a generative language model
+1. Given a query that we write, retrieve the most similar posts
+1. Plot the archive of articles by similarity (where similar posts are close together and different ones are far)
+1. Cluster the posts to identify the major common themes
+1. Extract major keywords from each cluster so we can identify what the clsuter is about
+1. (Experimental) Name clusters with a generative language model
## Setup
-
Let's start by installing the tools we'll need and then importing them.
-```python PYTHON
-!pip install cohere umap-learn altair annoy bertopic
-```
-```
-Requirement already satisfied: cohere in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (5.1.5)
-Requirement already satisfied: umap-learn in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (0.5.5)
-Requirement already satisfied: altair in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (5.2.0)
-Requirement already satisfied: annoy in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (1.17.3)
-Requirement already satisfied: bertopic in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (0.16.0)
-Requirement already satisfied: httpx>=0.21.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (0.27.0)
-Requirement already satisfied: pydantic>=1.9.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (2.6.0)
-Requirement already satisfied: typing_extensions>=4.0.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (4.10.0)
-Requirement already satisfied: numpy>=1.17 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.24.3)
-Requirement already satisfied: scipy>=1.3.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.11.1)
-Requirement already satisfied: scikit-learn>=0.22 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.3.0)
-Requirement already satisfied: numba>=0.51.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (0.57.0)
-Requirement already satisfied: pynndescent>=0.5 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (0.5.12)
-Requirement already satisfied: tqdm in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (4.65.0)
-Requirement already satisfied: jinja2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (3.1.2)
-Requirement already satisfied: jsonschema>=3.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (4.17.3)
-Requirement already satisfied: packaging in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (23.2)
-Requirement already satisfied: pandas>=0.25 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (2.0.3)
-Requirement already satisfied: toolz in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (0.12.0)
-Requirement already satisfied: hdbscan>=0.8.29 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (0.8.33)
-Requirement already satisfied: sentence-transformers>=0.4.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (2.6.1)
-Requirement already satisfied: plotly>=4.7.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (5.9.0)
-Requirement already satisfied: cython<3,>=0.27 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from hdbscan>=0.8.29->bertopic) (0.29.37)
-Requirement already satisfied: joblib>=1.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from hdbscan>=0.8.29->bertopic) (1.2.0)
-Requirement already satisfied: anyio in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (3.5.0)
-Requirement already satisfied: certifi in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (2023.11.17)
-Requirement already satisfied: httpcore==1.* in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (1.0.2)
-Requirement already satisfied: idna in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (3.4)
-Requirement already satisfied: sniffio in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (1.2.0)
-Requirement already satisfied: h11<0.15,>=0.13 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpcore==1.*->httpx>=0.21.2->cohere) (0.14.0)
-Requirement already satisfied: attrs>=17.4.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jsonschema>=3.0->altair) (22.1.0)
-Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jsonschema>=3.0->altair) (0.18.0)
-Requirement already satisfied: llvmlite<0.41,>=0.40.0dev0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from numba>=0.51.2->umap-learn) (0.40.0)
-Requirement already satisfied: python-dateutil>=2.8.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2.8.2)
-Requirement already satisfied: pytz>=2020.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2023.3.post1)
-Requirement already satisfied: tzdata>=2022.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2023.3)
-Requirement already satisfied: tenacity>=6.2.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from plotly>=4.7.0->bertopic) (8.2.2)
-Requirement already satisfied: annotated-types>=0.4.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pydantic>=1.9.2->cohere) (0.6.0)
-Requirement already satisfied: pydantic-core==2.16.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pydantic>=1.9.2->cohere) (2.16.1)
-Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from scikit-learn>=0.22->umap-learn) (2.2.0)
-Requirement already satisfied: transformers<5.0.0,>=4.32.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (4.39.3)
-Requirement already satisfied: torch>=1.11.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (2.2.2)
-Requirement already satisfied: huggingface-hub>=0.15.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (0.22.2)
-Requirement already satisfied: Pillow in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (10.0.1)
-Requirement already satisfied: MarkupSafe>=2.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jinja2->altair) (2.1.1)
-Requirement already satisfied: filelock in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (3.9.0)
-Requirement already satisfied: fsspec>=2023.5.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (2024.3.1)
-Requirement already satisfied: pyyaml>=5.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (6.0)
-Requirement already satisfied: requests in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (2.31.0)
-Requirement already satisfied: six>=1.5 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=0.25->altair) (1.16.0)
-Requirement already satisfied: sympy in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (1.11.1)
-Requirement already satisfied: networkx in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (3.1)
-Requirement already satisfied: regex!=2019.12.17 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (2022.7.9)
-Requirement already satisfied: tokenizers<0.19,>=0.14 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (0.15.2)
-Requirement already satisfied: safetensors>=0.4.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (0.4.2)
-Requirement already satisfied: charset-normalizer<4,>=2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (3.3.2)
-Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (1.26.18)
-Requirement already satisfied: mpmath>=0.19 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sympy->torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (1.3.0)
+```python
+!pip install cohere umap-learn altair annoy bertopic
```
-```python PYTHON
+ Requirement already satisfied: cohere in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (5.1.5)
+ Requirement already satisfied: umap-learn in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (0.5.5)
+ Requirement already satisfied: altair in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (5.2.0)
+ Requirement already satisfied: annoy in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (1.17.3)
+ Requirement already satisfied: bertopic in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (0.16.0)
+ Requirement already satisfied: httpx>=0.21.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (0.27.0)
+ Requirement already satisfied: pydantic>=1.9.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (2.6.0)
+ Requirement already satisfied: typing_extensions>=4.0.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from cohere) (4.10.0)
+ Requirement already satisfied: numpy>=1.17 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.24.3)
+ Requirement already satisfied: scipy>=1.3.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.11.1)
+ Requirement already satisfied: scikit-learn>=0.22 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (1.3.0)
+ Requirement already satisfied: numba>=0.51.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (0.57.0)
+ Requirement already satisfied: pynndescent>=0.5 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (0.5.12)
+ Requirement already satisfied: tqdm in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from umap-learn) (4.65.0)
+ Requirement already satisfied: jinja2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (3.1.2)
+ Requirement already satisfied: jsonschema>=3.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (4.17.3)
+ Requirement already satisfied: packaging in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (23.2)
+ Requirement already satisfied: pandas>=0.25 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (2.0.3)
+ Requirement already satisfied: toolz in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from altair) (0.12.0)
+ Requirement already satisfied: hdbscan>=0.8.29 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (0.8.33)
+ Requirement already satisfied: sentence-transformers>=0.4.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (2.6.1)
+ Requirement already satisfied: plotly>=4.7.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from bertopic) (5.9.0)
+ Requirement already satisfied: cython<3,>=0.27 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from hdbscan>=0.8.29->bertopic) (0.29.37)
+ Requirement already satisfied: joblib>=1.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from hdbscan>=0.8.29->bertopic) (1.2.0)
+ Requirement already satisfied: anyio in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (3.5.0)
+ Requirement already satisfied: certifi in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (2023.11.17)
+ Requirement already satisfied: httpcore==1.* in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (1.0.2)
+ Requirement already satisfied: idna in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (3.4)
+ Requirement already satisfied: sniffio in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpx>=0.21.2->cohere) (1.2.0)
+ Requirement already satisfied: h11<0.15,>=0.13 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from httpcore==1.*->httpx>=0.21.2->cohere) (0.14.0)
+ Requirement already satisfied: attrs>=17.4.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jsonschema>=3.0->altair) (22.1.0)
+ Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jsonschema>=3.0->altair) (0.18.0)
+ Requirement already satisfied: llvmlite<0.41,>=0.40.0dev0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from numba>=0.51.2->umap-learn) (0.40.0)
+ Requirement already satisfied: python-dateutil>=2.8.2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2.8.2)
+ Requirement already satisfied: pytz>=2020.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2023.3.post1)
+ Requirement already satisfied: tzdata>=2022.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->altair) (2023.3)
+ Requirement already satisfied: tenacity>=6.2.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from plotly>=4.7.0->bertopic) (8.2.2)
+ Requirement already satisfied: annotated-types>=0.4.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pydantic>=1.9.2->cohere) (0.6.0)
+ Requirement already satisfied: pydantic-core==2.16.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from pydantic>=1.9.2->cohere) (2.16.1)
+ Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from scikit-learn>=0.22->umap-learn) (2.2.0)
+ Requirement already satisfied: transformers<5.0.0,>=4.32.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (4.39.3)
+ Requirement already satisfied: torch>=1.11.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (2.2.2)
+ Requirement already satisfied: huggingface-hub>=0.15.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (0.22.2)
+ Requirement already satisfied: Pillow in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sentence-transformers>=0.4.1->bertopic) (10.0.1)
+ Requirement already satisfied: MarkupSafe>=2.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from jinja2->altair) (2.1.1)
+ Requirement already satisfied: filelock in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (3.9.0)
+ Requirement already satisfied: fsspec>=2023.5.0 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (2024.3.1)
+ Requirement already satisfied: pyyaml>=5.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (6.0)
+ Requirement already satisfied: requests in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (2.31.0)
+ Requirement already satisfied: six>=1.5 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=0.25->altair) (1.16.0)
+ Requirement already satisfied: sympy in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (1.11.1)
+ Requirement already satisfied: networkx in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (3.1)
+ Requirement already satisfied: regex!=2019.12.17 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (2022.7.9)
+ Requirement already satisfied: tokenizers<0.19,>=0.14 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (0.15.2)
+ Requirement already satisfied: safetensors>=0.4.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=0.4.1->bertopic) (0.4.2)
+ Requirement already satisfied: charset-normalizer<4,>=2 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (3.3.2)
+ Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers>=0.4.1->bertopic) (1.26.18)
+ Requirement already satisfied: mpmath>=0.19 in /Users/alexiscook/anaconda3/lib/python3.11/site-packages (from sympy->torch>=1.11.0->sentence-transformers>=0.4.1->bertopic) (1.3.0)
+
+
+
+```python
import cohere
import numpy as np
import pandas as pd
@@ -108,144 +114,167 @@ warnings.filterwarnings('ignore')
pd.set_option('display.max_colwidth', None)
```
-Fill in your Cohere API key in the next cell. To do this, begin by [signing up to Cohere](https://dashboard.cohere.com/) (for free!) if you haven't yet. Then get your API key [here](https://dashboard.cohere.com/api-keys).
+Fill in your Cohere API key in the next cell. To do this, begin by [signing up to Cohere](https://os.cohere.ai/) (for free!) if you haven't yet. Then get your API key [here](https://dashboard.cohere.com/api-keys).
+
-```python PYTHON
+```python
+# Paste your API key here. Remember to not share publicly
co = cohere.Client("COHERE_API_KEY") # Insert your Cohere API key
```
## Dataset: Top 3,000 Ask HN posts
-
We will use the top 3,000 posts from the Ask HN section of Hacker News. We provide a CSV containing the posts.
-```python PYTHON
+
+```python
+# Load the dataframe containing the text and metadata of the posts
df = pd.read_csv('https://storage.googleapis.com/cohere-assets/blog/text-clustering/data/askhn3k_df.csv', index_col=0)
print(f'Loaded a DataFrame with {len(df)} rows')
```
-```
-Loaded a DataFrame with 3000 rows
-```
+ Loaded a DataFrame with 3000 rows
+
-```python PYTHON
+
+```python
+# Let's glance at the contents of the dataframe with the text and metadata
df.head()
```
+
+
+
-
-
-
| - | title | -url | -text | -dead | -by | -score | -time | -timestamp | -type | -id | -parent | -descendants | -ranking | -deleted | -|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -I'm a software engineer going blind, how should I prepare? | -NaN | + +
| + | title | +url | +text | +dead | +by | +score | +time | +timestamp | +type | +id | +parent | +descendants | +ranking | +deleted | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | +I'm a software engineer going blind, how should I prepare? | +NaN | I'm a 24 y/o full stack engineer (I know some of you are rolling your eyes right now, just highlighting that I have experience on frontend apps as well as backend architecture). I've been working professionally for ~7 years building mostly javascript projects but also some PHP. Two years ago I was diagnosed with a condition called "Usher's Syndrome" - characterized by hearing loss, balance issues, and progressive vision loss.<p>I know there are blind software engineers out there. My main questions are:<p>- Are there blind frontend engineers?<p>- What kinds of software engineering lend themselves to someone with limited vision? Backend only?<p>- Besides a screen reader, what are some of the best tools for building software with limited vision?<p>- Does your company employ blind engineers? How well does it work? What kind of engineer are they?<p>I'm really trying to get ahead of this thing and prepare myself as my vision is degrading rather quickly. I'm not sure what I can do if I can't do SE as I don't have any formal education in anything. I've worked really hard to get to where I am and don't want it to go to waste.<p>Thank you for any input, and stay safe out there!<p>Edit:<p>Thank you all for your links, suggestions, and moral support, I really appreciate it. Since my diagnosis I've slowly developed a crippling anxiety centered around a feeling that I need to figure out the rest of my life before it's too late. I know I shouldn't think this way but it is hard not to. I'm very independent and I feel a pressure to "show up." I will look into these opportunities mentioned and try to get in touch with some more members of the blind engineering community. | -NaN | -zachrip | -3270 | -1587332026 | -2020-04-19 21:33:46+00:00 | -story | -22918980 | -NaN | -473.0 | -NaN | -NaN | -
| 1 | -Am I the longest-serving programmer – 57 years and counting? | -NaN | -In May of 1963, I started my first full-time job as a computer programmer for Mitchell Engineering Company, a supplier of steel buildings. At Mitchell, I developed programs in Fortran II on an IBM 1620 mostly to improve the efficiency of order processing and fulfillment. Since then, all my jobs for the past 57 years have involved computer programming. I am now a data scientist developing cloud-based big data fraud detection algorithms using machine learning and other advanced analytical technologies. Along the way, I earned a Master’s in Operations Research and a Master’s in Management Science, studied artificial intelligence for 3 years in a Ph.D. program for engineering, and just two years ago I received Graduate Certificates in Big Data Analytics from the schools of business and computer science at a local university (FAU). In addition, I currently hold the designation of Certified Analytics Professional (CAP). At 74, I still have no plans to retire or to stop programming. | -NaN | -genedangelo | -2634 | -1590890024 | -2020-05-31 01:53:44+00:00 | -story | -23366546 | -NaN | -531.0 | -NaN | -NaN | -
| 2 | -Is S3 down? | -NaN | -I'm getting<p>\{\n "errorCode" : "InternalError"\n}<p>When I attempt to use the AWS Console to view s3 | -NaN | -iamdeedubs | -2589 | -1488303958 | -2017-02-28 17:45:58+00:00 | -story | -13755673 | -NaN | -1055.0 | -NaN | -NaN | -
| 3 | -What tech job would let me get away with the least real work possible? | -NaN | +NaN | +zachrip | +3270 | +1587332026 | +2020-04-19 21:33:46+00:00 | +story | +22918980 | +NaN | +473.0 | +NaN | +NaN | +|
| 1 | +Am I the longest-serving programmer – 57 years and counting? | +NaN | +In May of 1963, I started my first full-time job as a computer programmer for Mitchell Engineering Company, a supplier of steel buildings. At Mitchell, I developed programs in Fortran II on an IBM 1620 mostly to improve the efficiency of order processing and fulfillment. Since then, all my jobs for the past 57 years have involved computer programming. I am now a data scientist developing cloud-based big data fraud detection algorithms using machine learning and other advanced analytical technologies. Along the way, I earned a Master’s in Operations Research and a Master’s in Management Science, studied artificial intelligence for 3 years in a Ph.D. program for engineering, and just two years ago I received Graduate Certificates in Big Data Analytics from the schools of business and computer science at a local university (FAU). In addition, I currently hold the designation of Certified Analytics Professional (CAP). At 74, I still have no plans to retire or to stop programming. | +NaN | +genedangelo | +2634 | +1590890024 | +2020-05-31 01:53:44+00:00 | +story | +23366546 | +NaN | +531.0 | +NaN | +NaN | +
| 2 | +Is S3 down? | +NaN | +I'm getting<p>{\n "errorCode" : "InternalError"\n}<p>When I attempt to use the AWS Console to view s3 | +NaN | +iamdeedubs | +2589 | +1488303958 | +2017-02-28 17:45:58+00:00 | +story | +13755673 | +NaN | +1055.0 | +NaN | +NaN | +
| 3 | +What tech job would let me get away with the least real work possible? | +NaN | Hey HN,<p>I'll probably get a lot of flak for this. Sorry.<p>I'm an average developer looking for ways to work as little as humanely possible.<p>The pandemic made me realize that I do not care about working anymore. The software I build is useless. Time flies real fast and I have to focus on my passions (which are not monetizable).<p>Unfortunately, I require shelter, calories and hobby materials. Thus the need for some kind of job.<p>Which leads me to ask my fellow tech workers, what kind of job (if any) do you think would fit the following requirements :<p>- No / very little involvement in the product itself (I do not care.)<p>- Fully remote (You can't do much when stuck in the office. Ideally being done in 2 hours in the morning then chilling would be perfect.)<p>- Low expectactions / vague job description.<p>- Salary can be on the lower side.<p>- No career advancement possibilities required. Only tech, I do not want to manage people.<p>- Can be about helping other developers, setting up infrastructure/deploy or pure data management since this is fun.<p>I think the only possible jobs would be some kind of backend-only dev or devops/sysadmin work. But I'm not sure these exist anymore, it seems like you always end up having to think about the product itself. Web dev jobs always required some involvement in the frontend.<p>Thanks for any advice (or hate, which I can't really blame you for). | -NaN | -lmueongoqx | -2022 | -1617784863 | -2021-04-07 08:41:03+00:00 | -story | -26721951 | -NaN | -1091.0 | -NaN | -NaN | -
| 4 | -What books changed the way you think about almost everything? | -NaN | -I was reflecting today about how often I think about Freakonomics. I don't study it religiously. I read it one time more than 10 years ago. I can only remember maybe a single specific anecdote from the book. And yet the simple idea that basically every action humans take can be traced back to an incentive has fundamentally changed the way I view the world. Can anyone recommend books that have had a similar impact on them? | -NaN | -anderspitman | -2009 | -1549387905 | -2019-02-05 17:31:45+00:00 | -story | -19087418 | -NaN | -1165.0 | -NaN | -NaN | -NaN | +lmueongoqx | +2022 | +1617784863 | +2021-04-07 08:41:03+00:00 | +story | +26721951 | +NaN | +1091.0 | +NaN | +NaN | + +
| 4 | +What books changed the way you think about almost everything? | +NaN | +I was reflecting today about how often I think about Freakonomics. I don't study it religiously. I read it one time more than 10 years ago. I can only remember maybe a single specific anecdote from the book. And yet the simple idea that basically every action humans take can be traced back to an incentive has fundamentally changed the way I view the world. Can anyone recommend books that have had a similar impact on them? | +NaN | +anderspitman | +2009 | +1549387905 | +2019-02-05 17:31:45+00:00 | +story | +19087418 | +NaN | +1165.0 | +NaN | +NaN | +
-
-
-
+
+
+
+
-## 2- Given a query that we write, retrieve the most similar posts
+
+## 2- Given a query that we write, retrieve the most similar posts
We're not limited to searching using existing items. If we get a query, we can embed it and find its nearest neighbors from the dataset.
-```python PYTHON
+
+```python
query = "How can I improve my knowledge of calculus?"
+# Get the query's embedding
+# We'll need to embed the query using the same model that we used to embed the post titles
+# so the query and titles are using the same embedding space.
query_embed = co.embed(texts=[query],
model="embed-english-v3.0",
truncate="RIGHT",
input_type="search_query").embeddings
+# Retrieve the nearest neighbors
similar_item_ids = search_index.get_nns_by_vector(query_embed[0], 10, include_distances=True)
+# Format & print the results
results = pd.DataFrame(data={'texts': df.iloc[similar_item_ids[0]]['title'],
'distance': similar_item_ids[1]})
print(f"Query:'{query}'\nNearest neighbors:")
results
```
-```
-Query:'How can I improve my knowledge of calculus?'
-Nearest neighbors:
-```
+ Query:'How can I improve my knowledge of calculus?'
+ Nearest neighbors:
+
+
+
+
| - | post titles | -distance | -
|---|---|---|
| 2991 | -Best Bank for Startups? | -0.883494 | -
| 2910 | -Who's looking for a cofounder? | -0.885087 | -
| 31 | -What startup/technology is on your 'to watch' list? | -0.887212 | -
| 685 | -What startup/technology is on your 'to watch' list? | -0.887212 | -
| 2123 | -Who is seeking a cofounder? | -0.889451 | -
| 727 | -Agriculture startups doing interesting work? | -0.899192 | -
| 2972 | -How should I evaluate a startup as I job hunt? | -0.901621 | -
| 2589 | -- What methods do you use to gain early customers for your startup? - | -0.903065 | -
| 2708 | -Is there VC appetite for defense related startups? | -0.904016 | -
| + | post titles | +distance | +
|---|---|---|
| 2991 | +Best Bank for Startups? | +0.883494 | +
| 2910 | +Who's looking for a cofounder? | +0.885087 | +
| 31 | +What startup/technology is on your 'to watch' list? | +0.887212 | +
| 685 | +What startup/technology is on your 'to watch' list? | +0.887212 | +
| 2123 | +Who is seeking a cofounder? | +0.889451 | +
| 727 | +Agriculture startups doing interesting work? | +0.899192 | +
| 2972 | +How should I evaluate a startup as I job hunt? | +0.901621 | +
| 2589 | +What methods do you use to gain early customers for your startup? | +0.903065 | +
| 2708 | +Is there VC appetite for defense related startups? | +0.904016 | +
-
-
-
+
+
+
+
-## 3- Plot the archive of articles by similarity
+
+## 3- Plot the archive of articles by similarity
What if we want to browse the archive instead of only searching it? Let's plot all the questions in a 2D chart so you're able to visualize the posts in the archive and their similarities.
-```python PYTHON
+
+```python
reducer = umap.UMAP(n_neighbors=100)
umap_embeds = reducer.fit_transform(embeds)
```
-```python PYTHON
+
+```python
df['x'] = umap_embeds[:,0]
df['y'] = umap_embeds[:,1]
+# Plot
chart = alt.Chart(df).mark_circle(size=60).encode(
x=#'x',
alt.X('x',
@@ -489,22 +568,44 @@ chart = alt.Chart(df).mark_circle(size=60).encode(
chart.interactive()
```
+
+
+
+
+
-## 4- Cluster the posts to identify the major common themes
+
+
+## 4- Cluster the posts to identify the major common themes
Let's proceed to cluster the embeddings using KMeans from scikit-learn.
-```python PYTHON
+
+```python
+# Pick the number of clusters
n_clusters = 8
+# Cluster the embeddings
kmeans_model = KMeans(n_clusters=n_clusters, random_state=0)
classes = kmeans_model.fit_predict(embeds)
```
## 5- Extract major keywords from each cluster so we can identify what the cluster is about
-```python PYTHON
+
+```python
+# Extract the keywords for each cluster
documents = df['title']
documents = pd.DataFrame({"Document": documents,
"ID": range(len(documents)),
@@ -516,7 +617,8 @@ count = count_vectorizer.transform(documents_per_topic.Document)
words = count_vectorizer.get_feature_names_out()
```
-```python PYTHON
+
+```python
ctfidf = ClassTfidfTransformer().fit_transform(count).toarray()
words_per_class = {label: [words[index] for index in ctfidf[label].argsort()[-10:]] for label in documents_per_topic.Topic}
df['cluster'] = classes
@@ -524,10 +626,10 @@ df['keywords'] = df['cluster'].map(lambda topic_num: ", ".join(np.array(words_pe
```
## Plot with clusters and keywords information
-
We can now plot the documents with their clusters and keywords
-```python PYTHON
+
+```python
selection = alt.selection_multi(fields=['keywords'], bind='legend')
chart = alt.Chart(df).transform_calculate(
@@ -562,10 +664,27 @@ chart = alt.Chart(df).transform_calculate(
chart.interactive()
```
+
+
+
+
+
-## 6- (Experimental) Naming clusters with a generative language model
+
+
+## 6- (Experimental) Naming clusters with a generative language model
While the extracted keywords do add a lot of information to help us identify the clusters at a glance, we should be able to have a generative model look at these keywords and suggest a name. So far I have reasonable results from a prompt that looks like this:
```
@@ -582,11 +701,9 @@ The common theme of the following words: | - | texts | -distance | -
|---|---|---|
| 2457 | -How do I improve my command of mathematical language? | -0.931286 | -
| 1235 | -How to learn new things better? | -1.024635 | -
| 145 | -How to self-learn math? | -1.044135 | -
| 1317 | -How can I learn to read mathematical notation? | -1.050976 | -
| 910 | -How Do You Learn? | -1.061253 | -
| 2432 | -How did you learn math notation? | -1.070800 | -
| 1994 | -How do I become smarter? | -1.083434 | -
| 1529 | -How do you personally learn? | -1.086088 | -
| 796 | -How do you keep improving? | -1.087251 | -
| 1286 | -How do I learn drawing? | -1.088468 | -
| + | texts | +distance | +
|---|---|---|
| 2457 | +How do I improve my command of mathematical language? | +0.931286 | +
| 1235 | +How to learn new things better? | +1.024635 | +
| 145 | +How to self-learn math? | +1.044135 | +
| 1317 | +How can I learn to read mathematical notation? | +1.050976 | +
| 910 | +How Do You Learn? | +1.061253 | +
| 2432 | +How did you learn math notation? | +1.070800 | +
| 1994 | +How do I become smarter? | +1.083434 | +
| 1529 | +How do you personally learn? | +1.086088 | +
| 796 | +How do you keep improving? | +1.087251 | +
| 1286 | +How do I learn drawing? | +1.088468 | +
+
We can’t wait to see what you start building! Share your projects or find support on our [Discord server](https://discord.com/invite/co-mmunity).
+
diff --git a/fern/pages/cookbooks/article-recommender-with-text-embeddings.mdx b/fern/pages/cookbooks/article-recommender-with-text-embeddings.mdx
index 29aa591c5..9507e35ac 100644
--- a/fern/pages/cookbooks/article-recommender-with-text-embeddings.mdx
+++ b/fern/pages/cookbooks/article-recommender-with-text-embeddings.mdx
@@ -1,32 +1,37 @@
---
-title: Article Recommender with Text Embedding and Classification
+title: Article Recommender with Text Embedding Classification Extraction
slug: /page/article-recommender-with-text-embeddings
description: "This page describes how to build a generative-AI tool to recommend articles with Cohere."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, AI assistant, recommendation engines"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
+
## Article Recommender with Text Embedding, Classification, and Extraction
-This is a simple demonstration of how we can stack multiple NLP models together
+This is a simple demonstration of how we can stack multiple NLP models together
to get an output much closer to our desired outcome.
-Embeddings can capture the meaning of a piece of text beyond keyword-matching. In this article, we will build a simple news article recommender system that computes the embeddings of all available articles and recommend the most relevant articles based on embeddings similarity.
+Embeddings can capture the meaning of a piece of text beyond keyword-matching. In this article, we will build a simple news article recommender system that computes the embeddings of all available articles and recommend the most relevant articles based on embeddings similarity.
-We will also make the recommendation tighter by using text classification to recommend only articles within the same category. We will then extract a list of tags from each recommended article, which can further help readers discover new articles.
+We will also make the recommendation tighter by using text classification to recommend only articles within the same category. We will then extract a list of tags from each recommended article, which can further help readers discover new articles.
All this will be done via three Cohere API endpoints stacked together: Embed, Classify, and Chat.
-
+
+
+
We will implement the following steps:
@@ -38,25 +43,26 @@ We will implement the following steps:
**4: Show the top 5 recommended articles.**
-```python PYTHON
+
+```python
! pip install cohere
```
-```txt title="Output"
-Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
-Collecting cohere
- Downloading cohere-1.3.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
-[K |████████████████████████████████| 18.0 MB 135 kB/s
-[?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from cohere) (2.23.0)
-Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (1.24.3)
-Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2022.6.15)
-Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2.10)
-Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (3.0.4)
-Installing collected packages: cohere
-Successfully installed cohere-1.3.10
-```
+ Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
+ Collecting cohere
+ Downloading cohere-1.3.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
+ [K |████████████████████████████████| 18.0 MB 135 kB/s
+ [?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from cohere) (2.23.0)
+ Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (1.24.3)
+ Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2022.6.15)
+ Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2.10)
+ Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (3.0.4)
+ Installing collected packages: cohere
+ Successfully installed cohere-1.3.10
-```python PYTHON
+
+
+```python
import numpy as np
import pandas as pd
import re
@@ -66,70 +72,96 @@ co = cohere.Client("COHERE_API_KEY") # Get your API key: https://dashboard.coher
```
-
+
+
## 1.1: Get articles
-Throughout this article, we'll use the [BBC news article dataset](https://www.kaggle.com/competitions/learn-ai-bbc/data?select=BBC+News+Train.csv) as an example [\[Source\]](http://mlg.ucd.ie/datasets/bbc.html). This dataset consists of articles from a few categories: business, politics, tech, entertainment, and sport.
+Throughout this article, we'll use the [BBC news article dataset](https://www.kaggle.com/competitions/learn-ai-bbc/data?select=BBC+News+Train.csv) as an example [[Source]](http://mlg.ucd.ie/datasets/bbc.html). This dataset consists of articles from a few categories: business, politics, tech, entertainment, and sport.
We'll extract a subset of the data and in Step 1, use the first 100 data points.
-```python PYTHON
+
+
+
+```python
+# Load the dataset to a dataframe
df = pd.read_csv('https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/notebooks/data/bbc_news_subset.csv', delimiter=',')
+# Select a portion of the dataset
INP_START = 0
INP_END = 100
df_inputs = df.iloc[INP_START:INP_END]
df_inputs = df_inputs.copy()
+# Remove columns we don't need
df_inputs.drop(['ArticleId','Category'],axis=1,inplace=True)
+# View the data
df_inputs.head()
```
+
+
+
-
-
-
+
+
+
+
+
+
## 1.2: Turn articles into embeddings
+
Next we turn each article text into embeddings. An [embedding](https://docs.cohere.ai/embedding-wiki) is a list of numbers that our models use to represent a piece of text, capturing its context and meaning.
We do this by calling Cohere's [Embed endpoint](https://docs.cohere.ai/embed-reference), which takes in text as input and returns embeddings as output.
-```python PYTHON
+
+```python
+# Get text embeddings via the Embed endpoint
articles = df_inputs['Text'].tolist()
output = co.embed(
@@ -141,31 +173,36 @@ embeds = output.embeddings
print('Number of articles:', len(embeds))
```
-```
-Number of articles: 100
-```
+ Number of articles: 100
+
## 1.3: Pick one article and find the most similar articles
Next, we pick any one article to be the one the reader is currently reading (let's call this the target) and find other articles with the most similar embeddings (let's call these candidates) using cosine similarity.
-[Cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) is a metric that measures how similar sequences of numbers are (embeddings in our case), and we compute it for each target-candidate pair.
+[Cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) is a metric that measures how similar sequences of numbers are (embeddings in our case), and we compute it for each target-candidate pair.
+
-```python PYTHON
+```python
+# Choose one article ID as the one you are currently reading
print(f'Choose one article ID between {INP_START} and {INP_END-1} below...')
```
-```
-Choose one article ID between 0 and 99 below...
-```
+ Choose one article ID between 0 and 99 below...
+
+
-```python PYTHON
+```python
+# Enter your article ID
READING_IDX = 70
+# Get embedding for the article
reading = embeds[READING_IDX]
```
-```python PYTHON
+
+```python
+# Calculate cosine similarity between the target and candidate articles
from sklearn.metrics.pairwise import cosine_similarity
@@ -186,9 +223,12 @@ def get_similarity(target,candidates):
return similarity_scores
```
-```python PYTHON
+
+```python
+# Get the similarity between the target and candidate articles
similarity = get_similarity(reading,embeds)
+# View the top 5 articles
print('Target:')
print(f'[ID {READING_IDX}]',df_inputs['Text'][READING_IDX][:100],'...','\n')
@@ -197,88 +237,113 @@ for i in similarity[1:6]: # Exclude the target article
print(f'[ID {i[0]}]',df_inputs['Text'][i[0]][:100],'...')
```
-```
-Target:
-[ID 70] aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanis ...
-
-Candidates:
-[ID 23] ferguson urges henry punishment sir alex ferguson has called on the football association to punish a ...
-[ID 51] mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and ...
-[ID 73] balco case trial date pushed back the trial date for the bay area laboratory cooperative (balco) ste ...
-[ID 41] mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after ...
-[ID 42] premier league planning cole date the premier league is attempting to find a mutually convenient dat ...
-```
+ Target:
+ [ID 70] aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanis ...
+
+ Candidates:
+ [ID 23] ferguson urges henry punishment sir alex ferguson has called on the football association to punish a ...
+ [ID 51] mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and ...
+ [ID 73] balco case trial date pushed back the trial date for the bay area laboratory cooperative (balco) ste ...
+ [ID 41] mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after ...
+ [ID 42] premier league planning cole date the premier league is attempting to find a mutually convenient dat ...
+
+
+# **2: Keep only articles of the same category using text classification**
-| - | Text | -
|---|---|
| 0 | -worldcom ex-boss launches defence lawyers defe... | -
| 1 | -german business confidence slides german busin... | -
| 2 | -bbc poll indicates economic gloom citizens in ... | -
| 3 | -lifestyle governs mobile choice faster bett... | -
| 4 | -enron bosses in $168m payout eighteen former e... | -
| + | Text | +
|---|---|
| 0 | +worldcom ex-boss launches defence lawyers defe... | +
| 1 | +german business confidence slides german busin... | +
| 2 | +bbc poll indicates economic gloom citizens in ... | +
| 3 | +lifestyle governs mobile choice faster bett... | +
| 4 | +enron bosses in $168m payout eighteen former e... | +
+
Two articles may be similar but they may not necessarily belong to the same category. For example, an article about a sports team manager facing a fine may be similar to another about a business entity facing a fine, but they are not of the same category.
Perhaps we can make the system better by only recommending articles of the same category. For this, let's build a news category classifier.
+
+
## 2.1: Build a classifier
-We use Cohere's [Classify endpoint](https://docs.cohere.com/reference/classify) to build a news category classifier, classifying articles into five categories: Business, Politics, Tech, Entertainment, and Sport.
+
+We use Cohere's [Classify endpoint](https://docs.cohere.ai/classify-reference) to build a news category classifier, classifying articles into five categories: Business, Politics, Tech, Entertainment, and Sport.
A typical text classification model requires hundreds/thousands of data points to train, but with this endpoint, we can build a classifier with a few as five examples per class.
To build the classifier, we need a set of examples consisting of text (news text) and labels (news category). The BBC News dataset happens to have both (columns 'Text' and 'Category'), so this time we’ll use the categories for building our examples. For this, we will set aside another portion of dataset.
-```python PYTHON
+
+```python
+# Select a portion of the dataset to sample the classification examples from
EX_START = 100
EX_END = 200
df_examples = df.iloc[EX_START:EX_END]
df_examples = df_examples.copy()
+# Remove columns we don't need
df_examples.drop(['ArticleId'],axis=1,inplace=True)
+# View the data
df_examples.head()
```
+
+
+
-
-
-
+
+
+
+
+
+
With the Classify endpoint, there is a limit of 512 tokens per input. This means full articles won't be able to fit in the examples, so we will approximate and limit each article to its first 300 characters.
-```python PYTHON
+
+```python
+# Shorten the example articles (because the medium endpoint max token limit is 2048)
MAX_CHARS = 300
def shorten_text(text):
@@ -289,11 +354,15 @@ df_examples['Text'] = df_examples['Text'].apply(shorten_text)
The Classify endpoint needs a minimum of 2 examples for each category. We'll have 5 examples each, sampled randomly from the dataset. We have 5 categories, so we will have a total of 25 examples.
-```python PYTHON
-EX_PER_CAT = 5
+```python
+# Set the number of examples per category
+EX_PER_CAT = 5
+
+# Get the list of all available categories
categories = df_examples['Category'].unique().tolist()
+# Create list of examples containing texts and labels
ex_texts = []
ex_labels = []
for category in categories:
@@ -308,23 +377,26 @@ print(f'Number of categories: {len(categories)}')
print(f'Total number of examples: {len(ex_texts)}')
```
-```
-Number of examples per category: 5
-List of categories: ['business', 'politics', 'tech', 'entertainment', 'sport']
-Number of categories: 5
-Total number of examples: 25
-```
+ Number of examples per category: 5
+ List of categories: ['business', 'politics', 'tech', 'entertainment', 'sport']
+ Number of categories: 5
+ Total number of examples: 25
+
Once the examples are ready, we can now get the classifications. Here is a function that returns the classification given an input.
-```python PYTHON
+
+```python
+# Get classifications via the Classify endpoint
from cohere import ClassifyExample
+# Collate the examples
examples = []
for txt, lbl in zip(ex_texts,ex_labels):
examples.append(ClassifyExample(text=txt, label=lbl))
+# Classification function
def classify_text(texts, examples):
classifications = co.classify(
inputs=texts,
@@ -338,97 +410,126 @@ def classify_text(texts, examples):
Before actually using the classifier, let's first test its performance. Here we take another 100 data points as the test dataset and the classifier will predict its class i.e. news category.
-```python PYTHON
+
+```python
+# Select a portion of the dataset for testing the classifier
TEST_START = 200
TEST_END = 300
df_test = df.iloc[TEST_START:TEST_END]
df_test = df_test.copy()
+# Remove columns we don't need
df_test.drop(['ArticleId'],axis=1,inplace=True)
+# Shorten the text to fit token limit
df_test['Text'] = df_test['Text'].apply(shorten_text)
+# View the data
df_test.head()
```
+
+
+
| - | Text | -Category | -
|---|---|---|
| 100 | -honda wins china copyright ruling japan s hond... | -business | -
| 101 | -ukip could sue veritas defectors the uk indepe... | -politics | -
| 102 | -security warning over fbi virus the us feder... | -tech | -
| 103 | -europe backs digital tv lifestyle how people r... | -tech | -
| 104 | -celebrities get to stay in jungle all four con... | -entertainment | -
| + | Text | +Category | +
|---|---|---|
| 100 | +honda wins china copyright ruling japan s hond... | +business | +
| 101 | +ukip could sue veritas defectors the uk indepe... | +politics | +
| 102 | +security warning over fbi virus the us feder... | +tech | +
| 103 | +europe backs digital tv lifestyle how people r... | +tech | +
| 104 | +celebrities get to stay in jungle all four con... | +entertainment | +
-
-
-
+
+
+
+
-```python PYTHON
+
+
+
+```python
+# Create batches of texts and classify them
predictions = []
BATCH_SIZE = 90 # The API accepts a maximum of 96 inputs
for i in range(0, len(df_test['Text']), BATCH_SIZE):
batch_texts = df_test['Text'][i:i+BATCH_SIZE].tolist()
- predictions.extend(classify_text(batch_texts, examples))
-
+ predictions.extend(classify_text(batch_texts, examples))
+
+# Actual classes
actual = df_test['Category'].tolist()
```
-```python PYTHON
+
+```python
+# Compute metrics on the test dataset
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(actual, predictions)
print(f'Accuracy: {accuracy*100}')
```
-```
-Accuracy: 89.0
-```
+ Accuracy: 89.0
+
-We get a good accuracy score of 91%, so the classifier is ready to be
+We get a good accuracy score of 91%, so the classifier is ready to be
implemented in our recommender system.
-| - | Text | -Category | -
|---|---|---|
| 200 | -sa return to mauritius top seeds south africa ... | -sport | -
| 201 | -snow patrol feted at irish awards snow patrol ... | -entertainment | -
| 202 | -clyde 0-5 celtic celtic brushed aside clyde to... | -sport | -
| 203 | -bad weather hits nestle sales a combination of... | -business | -
| 204 | -net fingerprints combat attacks eighty large n... | -tech | -
| + | Text | +Category | +
|---|---|---|
| 200 | +sa return to mauritius top seeds south africa ... | +sport | +
| 201 | +snow patrol feted at irish awards snow patrol ... | +entertainment | +
| 202 | +clyde 0-5 celtic celtic brushed aside clyde to... | +sport | +
| 203 | +bad weather hits nestle sales a combination of... | +business | +
| 204 | +net fingerprints combat attacks eighty large n... | +tech | +
+# **3: Extract tags from these articles.**
-We now proceed to the tags extraction step. Compared to the previous two steps, this step is not about sorting or filtering articles, but rather enriching them with more information.
+
+
+We now proceed to the tags extraction step. Compared to the previous two steps, this step is not about sorting or filtering articles, but rather enriching them with more information.
We do this with the Chat endpoint.
We call the endpoint by specifying a few settings, and it will generate the corresponding extractions.
-```python PYTHON
+
+```python
+# Get extractions via the Generate endpoint
def extract_tags(article):
prompt = f"""Given an article, extract a list of tags containing keywords of that article.
+# Examples
Article: japanese banking battle at an end japan s sumitomo mitsui \
financial has withdrawn its takeover offer for rival bank ufj holdings enabling the \
latter to merge with mitsubishi tokyo. sumitomo bosses told counterparts at ufj of its \
@@ -454,8 +555,8 @@ Tags: apple, apple powerbook 100, laptop
Article:{article}
Tags:"""
-
-
+
+
response = co.chat(
model='command-r',
message=prompt,
@@ -464,39 +565,48 @@ Tags:"""
return response.text
```
-
+# **4: Show the top 5 recommended articles.**
+
+
Let's now put everything together for our article recommender system.
First, we select the target article and compute the similarity scores against the candidate articles.
-```python PYTHON
+
+```python
+# Choose one article ID as the one you are currently reading
print(f'Choose one article ID between {INP_START} and {INP_END-1} below...')
```
-```
-Choose one article ID between 0 and 99 below...
-```
+ Choose one article ID between 0 and 99 below...
+
-```python PYTHON
+
+```python
+# Enter your article ID
READING_IDX = 70
+# Get embedding for the article
reading = embeds[READING_IDX]
+# Get the similarity between the target and candidate articles
similarity = get_similarity(reading,embeds)
```
Next, we filter the articles via classification. Finally, we extract the keywords from each article and show the recommendations.
-```python PYTHON
+
+
+```python
+# Choose the number of articles to recommend
SHOW_TOP = 5
+# Shorten the text to fit token limit
df_inputs = df_inputs.copy()
df_inputs['Text'] = df_inputs['Text'].apply(shorten_text)
+# Get the recommendations
def get_recommendations(reading_idx,similarity,show_top):
# Show the current article
@@ -515,7 +625,7 @@ def get_recommendations(reading_idx,similarity,show_top):
# Classify each candidate article
candidate_class = classify_text([df_inputs['Text'][idx]],examples)
-
+
# Show recommendations
if target_class == candidate_class and idx != reading_idx:
selection = df_inputs['Text'][idx][:MAX_CHARS]
@@ -526,7 +636,7 @@ def get_recommendations(reading_idx,similarity,show_top):
if tags:
print(f'Tags: {tags.strip()}\n')
else:
- print(f'Tags: none\n')
+ print(f'Tags: none\n')
# Increment the article count
count += 1
@@ -536,109 +646,122 @@ def get_recommendations(reading_idx,similarity,show_top):
break
```
-```python PYTHON
-get_recommendations(READING_IDX,similarity,SHOW_TOP)
-```
+```python
+# Show the recommended articles
+get_recommendations(READING_IDX,similarity,SHOW_TOP)
```
------- You are reading... ------
-[ID 70] Article: aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanish football federation for his comments about thierry henry. the 66-year-old criticised his 3000 euros (£2 060) punishment even though it was far below the maximum penalty. i am not guilty nor do i ...
------- You might also like... ------
-[ID 23] Article: ferguson urges henry punishment sir alex ferguson has called on the football association to punish arsenal s thierry henry for an incident involving gabriel heinze. ferguson believes henry deliberately caught heinze on the head with his knee during united s controversial win. the united boss said i...
-Tags: football, sir alex ferguson, thierry henry, arsenal, manchester united
+ ------ You are reading... ------
+ [ID 70] Article: aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanish football federation for his comments about thierry henry. the 66-year-old criticised his 3000 euros (£2 060) punishment even though it was far below the maximum penalty. i am not guilty nor do i ...
+
+ ------ You might also like... ------
+ [ID 23] Article: ferguson urges henry punishment sir alex ferguson has called on the football association to punish arsenal s thierry henry for an incident involving gabriel heinze. ferguson believes henry deliberately caught heinze on the head with his knee during united s controversial win. the united boss said i...
+ Tags: football, sir alex ferguson, thierry henry, arsenal, manchester united
+
+ [ID 51] Article: mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and arsene wenger would swap places with him. mourinho s side were knocked out of the fa cup by newcastle last sunday before seeing barcelona secure a 2-1 champions league first-leg lead in the nou camp....
+ Tags: chelsea, jose mourinho, sir alex ferguson, arsene wenger, fa cup, newcastle, barcelona, champions league
+
+ [ID 41] Article: mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after their disastrous uefa cup exit at the hands of auxerre at ibrox on wednesday. mcleish told bbc radio five live: we were in pole position to get through to the next stage but we blew it we absolutel...
+ Tags: rangers, alex mcleish, auxerre, uefa cup, ibrox
+
+ [ID 42] Article: premier league planning cole date the premier league is attempting to find a mutually convenient date to investigate allegations chelsea made an illegal approach for ashley cole. both chelsea and arsenal will be asked to give evidence to a premier league commission but no deadline has been put on ...
+ Tags: premier league, chelsea, arsenal, ashley cole
+
+ [ID 14] Article: ireland 21-19 argentina an injury-time dropped goal by ronan o gara stole victory for ireland from underneath the noses of argentina at lansdowne road on saturday. o gara kicked all of ireland s points with two dropped goals and five penalties to give the home side a 100% record in their autumn i...
+ Tags: rugby, ireland, argentina, ronan o gara
+
-[ID 51] Article: mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and arsene wenger would swap places with him. mourinho s side were knocked out of the fa cup by newcastle last sunday before seeing barcelona secure a 2-1 champions league first-leg lead in the nou camp....
-Tags: chelsea, jose mourinho, sir alex ferguson, arsene wenger, fa cup, newcastle, barcelona, champions league
-[ID 41] Article: mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after their disastrous uefa cup exit at the hands of auxerre at ibrox on wednesday. mcleish told bbc radio five live: we were in pole position to get through to the next stage but we blew it we absolutel...
-Tags: rangers, alex mcleish, auxerre, uefa cup, ibrox
+Keeping to the Section 1.3 example, here we see how the classification and extraction steps have improved our recommendation outcome.
-[ID 42] Article: premier league planning cole date the premier league is attempting to find a mutually convenient date to investigate allegations chelsea made an illegal approach for ashley cole. both chelsea and arsenal will be asked to give evidence to a premier league commission but no deadline has been put on ...
-Tags: premier league, chelsea, arsenal, ashley cole
+First, now the article with ID 73 (non sport) doesn't get recommended anymore. And now we have the tags related to each article being generated.
-[ID 14] Article: ireland 21-19 argentina an injury-time dropped goal by ronan o gara stole victory for ireland from underneath the noses of argentina at lansdowne road on saturday. o gara kicked all of ireland s points with two dropped goals and five penalties to give the home side a 100% record in their autumn i...
-Tags: rugby, ireland, argentina, ronan o gara
-```
-Keeping to the Section 1.3 example, here we see how the classification and extraction steps have improved our recommendation outcome.
-First, now the article with ID 73 (non sport) doesn't get recommended anymore. And now we have the tags related to each article being generated.
Let's try a couple of other articles in business and tech and see the output...
Business article (returning recommendations around German economy and economic growth/slump):
-```python PYTHON
+```python
+# A business news article example (ID 40)
+
+# Enter your article ID
READING_IDX = 1
+# Get embedding for the article
reading = embeds[READING_IDX]
+# Get the similarity between the target and candidate articles
similarity = get_similarity(reading,embeds)
+# Show the recommended articles
get_recommendations(READING_IDX,similarity,SHOW_TOP)
```
-```
------- You are reading... ------
-[ID 1] Article: german business confidence slides german business confidence fell in february knocking hopes of a speedy recovery in europe s largest economy. munich-based research institute ifo said that its confidence index fell to 95.5 in february from 97.5 in january its first decline in three months. the stu...
-
------- You might also like... ------
-[ID 56] Article: borussia dortmund near bust german football club and former european champion borussia dortmund has warned it will go bankrupt if rescue talks with creditors fail. the company s shares tumbled after it said it has entered a life-threatening profitability and financial situation . borussia dortmund...
-Tags: borussia dortmund, german football, bankruptcy
-
-[ID 2] Article: bbc poll indicates economic gloom citizens in a majority of nations surveyed in a bbc world service poll believe the world economy is worsening. most respondents also said their national economy was getting worse. but when asked about their own family s financial outlook a majority in 14 countries...
-Tags: bbc, economy, financial outlook
-
-[ID 8] Article: car giant hit by mercedes slump a slump in profitability at luxury car maker mercedes has prompted a big drop in profits at parent daimlerchrysler. the german-us carmaker saw fourth quarter operating profits fall to 785m euros ($1bn) from 2.4bn euros in 2003. mercedes-benz s woes - its profits slid...
-Tags: daimlerchrysler, mercedes, luxury car, profitability
+ ------ You are reading... ------
+ [ID 1] Article: german business confidence slides german business confidence fell in february knocking hopes of a speedy recovery in europe s largest economy. munich-based research institute ifo said that its confidence index fell to 95.5 in february from 97.5 in january its first decline in three months. the stu...
+
+ ------ You might also like... ------
+ [ID 56] Article: borussia dortmund near bust german football club and former european champion borussia dortmund has warned it will go bankrupt if rescue talks with creditors fail. the company s shares tumbled after it said it has entered a life-threatening profitability and financial situation . borussia dortmund...
+ Tags: borussia dortmund, german football, bankruptcy
+
+ [ID 2] Article: bbc poll indicates economic gloom citizens in a majority of nations surveyed in a bbc world service poll believe the world economy is worsening. most respondents also said their national economy was getting worse. but when asked about their own family s financial outlook a majority in 14 countries...
+ Tags: bbc, economy, financial outlook
+
+ [ID 8] Article: car giant hit by mercedes slump a slump in profitability at luxury car maker mercedes has prompted a big drop in profits at parent daimlerchrysler. the german-us carmaker saw fourth quarter operating profits fall to 785m euros ($1bn) from 2.4bn euros in 2003. mercedes-benz s woes - its profits slid...
+ Tags: daimlerchrysler, mercedes, luxury car, profitability
+
+ [ID 32] Article: china continues rapid growth china s economy has expanded by a breakneck 9.5% during 2004 faster than predicted and well above 2003 s 9.1%. the news may mean more limits on investment and lending as beijing tries to take the economy off the boil. china has sucked in raw materials and energy to fee...
+ Tags: china, economy, beijing
+
+ [ID 96] Article: bmw to recall faulty diesel cars bmw is to recall all cars equipped with a faulty diesel fuel-injection pump supplied by parts maker robert bosch. the faulty part does not represent a safety risk and the recall only affects pumps made in december and january. bmw said that it was too early to say h...
+ Tags: bmw, diesel cars, robert bosch, fuel injection pump
+
-[ID 32] Article: china continues rapid growth china s economy has expanded by a breakneck 9.5% during 2004 faster than predicted and well above 2003 s 9.1%. the news may mean more limits on investment and lending as beijing tries to take the economy off the boil. china has sucked in raw materials and energy to fee...
-Tags: china, economy, beijing
-
-[ID 96] Article: bmw to recall faulty diesel cars bmw is to recall all cars equipped with a faulty diesel fuel-injection pump supplied by parts maker robert bosch. the faulty part does not represent a safety risk and the recall only affects pumps made in december and january. bmw said that it was too early to say h...
-Tags: bmw, diesel cars, robert bosch, fuel injection pump
-```
Tech article (returning recommendations around consumer devices):
-```python PYTHON
+```python
+# A tech news article example (ID 30)
+
+# Enter your article ID
READING_IDX = 71
+# Get embedding for the article
reading = embeds[READING_IDX]
+# Get the similarity between the target and candidate articles
similarity = get_similarity(reading,embeds)
+# Show the recommended articles
get_recommendations(READING_IDX,similarity,SHOW_TOP)
```
-```txt title="Output"
------- You are reading... ------
-[ID 71] Article: camera phones are must-haves four times more mobiles with cameras in them will be sold in europe by the end of 2004 than last year says a report from analysts gartner. globally the number sold will reach 159 million an increase of 104%. the report predicts that nearly 70% of all mobile phones ...
-
------- You might also like... ------
-[ID 3] Article: lifestyle governs mobile choice faster better or funkier hardware alone is not going to help phone firms sell more handsets research suggests. instead phone firms keen to get more out of their customers should not just be pushing the technology for its own sake. consumers are far more interest...
-Tags: mobile, lifestyle, phone firms, handsets
-
-[ID 69] Article: gates opens biggest gadget fair bill gates has opened the consumer electronics show (ces) in las vegas saying that gadgets are working together more to help people manage multimedia content around the home and on the move. mr gates made no announcement about the next generation xbox games console ...
-Tags: bill gates, consumer electronics show, gadgets, xbox
-
-[ID 46] Article: china ripe for media explosion asia is set to drive global media growth to 2008 and beyond with china and india filling the two top spots analysts have predicted. japan south korea and singapore will also be strong players but china s demographics give it the edge a media conference in londo...
-Tags: china, india, japan, south korea, singapore, global media growth
+ ------ You are reading... ------
+ [ID 71] Article: camera phones are must-haves four times more mobiles with cameras in them will be sold in europe by the end of 2004 than last year says a report from analysts gartner. globally the number sold will reach 159 million an increase of 104%. the report predicts that nearly 70% of all mobile phones ...
+
+ ------ You might also like... ------
+ [ID 3] Article: lifestyle governs mobile choice faster better or funkier hardware alone is not going to help phone firms sell more handsets research suggests. instead phone firms keen to get more out of their customers should not just be pushing the technology for its own sake. consumers are far more interest...
+ Tags: mobile, lifestyle, phone firms, handsets
+
+ [ID 69] Article: gates opens biggest gadget fair bill gates has opened the consumer electronics show (ces) in las vegas saying that gadgets are working together more to help people manage multimedia content around the home and on the move. mr gates made no announcement about the next generation xbox games console ...
+ Tags: bill gates, consumer electronics show, gadgets, xbox
+
+ [ID 46] Article: china ripe for media explosion asia is set to drive global media growth to 2008 and beyond with china and india filling the two top spots analysts have predicted. japan south korea and singapore will also be strong players but china s demographics give it the edge a media conference in londo...
+ Tags: china, india, japan, south korea, singapore, global media growth
+
+ [ID 19] Article: moving mobile improves golf swing a mobile phone that recognises and responds to movements has been launched in japan. the motion-sensitive phone - officially titled the v603sh - was developed by sharp and launched by vodafone s japanese division. devised mainly for mobile gaming users can also ac...
+ Tags: mobile phone, japan, sharp, vodafone, golf swing
+
+ [ID 63] Article: what high-definition will do to dvds first it was the humble home video then it was the dvd and now hollywood is preparing for the next revolution in home entertainment - high-definition. high-definition gives incredible 3d-like pictures and surround sound. the dvd disks and the gear to play the...
+ Tags: high-definition, dvd, hollywood, home entertainment
+
-[ID 19] Article: moving mobile improves golf swing a mobile phone that recognises and responds to movements has been launched in japan. the motion-sensitive phone - officially titled the v603sh - was developed by sharp and launched by vodafone s japanese division. devised mainly for mobile gaming users can also ac...
-Tags: mobile phone, japan, sharp, vodafone, golf swing
-
-[ID 63] Article: what high-definition will do to dvds first it was the humble home video then it was the dvd and now hollywood is preparing for the next revolution in home entertainment - high-definition. high-definition gives incredible 3d-like pictures and surround sound. the dvd disks and the gear to play the...
-Tags: high-definition, dvd, hollywood, home entertainment
-```
In conclusion, this demonstrates an example of how we can stack multiple NLP endpoints together to get an output much closer to our desired outcome.
In practice, hosting and maintaining multiple models can turn quickly into a complex activity. But by leveraging Cohere endpoints, this task is reduced to a simple API call.
-
-```
-
-```
diff --git a/fern/pages/cookbooks/basic-multi-step.mdx b/fern/pages/cookbooks/basic-multi-step.mdx
index 2f45b1bfb..739a2c12b 100644
--- a/fern/pages/cookbooks/basic-multi-step.mdx
+++ b/fern/pages/cookbooks/basic-multi-step.mdx
@@ -1,16 +1,19 @@
---
-title: Multi-Step Tool Use with Cohere
+title: Multi-Step Tool Use
slug: /page/basic-multi-step
description: "This page describes how to create a multi-step, tool-using AI agent with Cohere's tool use functionality."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, AI assistant, agent, LLMs, agent tool use"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
+
In this notebook, we'll build a simple semantic search engine. The applications of semantic search go beyond building a web search engine. They can empower a private search engine for internal documents or records. It can also be used to power features like StackOverflow's "similar questions" feature.
1. Get the archive of questions
2. [Embed](https://docs.cohere.ai/embed-reference/) the archive
3. Search using an index and nearest neighbor search
-4. Visualize the archive based on the embeddings
+4. Visualize the archive based on the embeddings
+
+
+```python
+# Install Cohere for embeddings, Umap to reduce embeddings to 2 dimensions,
+# Altair for visualization, Annoy for approximate nearest neighbor search
+# TODO: upgrade to "cohere>5"!pip install "cohere<5" umap-learn altair annoy datasets tqdm
+```
And if you're running an older version of the SDK, you might need to upgrade it like so:
-```python PYTHON
+
+```python
#!pip install --upgrade cohere
```
-Get your Cohere API key by [signing up here](https://dashboard.cohere.com/register). Paste it in the cell below.
+Get your Cohere API key by [signing up here](https://os.cohere.ai/register). Paste it in the cell below.
+
+## 1. Getting Set up
-## 1. Getting Set Up
-```python PYTHON
+```python
#@title Import libraries (Run this cell to execute required code) {display-mode: "form"}
import cohere
@@ -54,144 +62,169 @@ warnings.filterwarnings('ignore')
pd.set_option('display.max_colwidth', None)
```
-You'll need your API key for this next cell. [Sign up to Cohere](https://dashboard.cohere.com/) and get one if you haven't yet.
+You'll need your API key for this next cell. [Sign up to Cohere](https://os.cohere.ai/) and get one if you haven't yet.
-```python PYTHON
+
+```python
+# We'll set up the name of the model we want to use, the API key, and the input type.
+# Create and retrieve a Cohere API key from dashboard.cohere.ai/welcome/register
+# Paste your API key here. Remember to not share publicly
model_name = "embed-english-v3.0"
api_key = ""
input_type_embed = "search_document"
+# Now we'll set up the cohere client.
co = cohere.Client(api_key)
```
## 2. Get The Archive of Questions
-
We'll use the [trec](https://www.tensorflow.org/datasets/catalog/trec) dataset which is made up of questions and their categories.
-```python PYTHON
+
+```python
+# Get dataset
dataset = load_dataset("trec", split="train")
+# Import into a pandas dataframe, take only the first 1000 rows
df = pd.DataFrame(dataset)[:1000]
+# Preview the data to ensure it has loaded correctly
df.head(10)
```
+
+
+
-
-
-
+
+
+
+
-## 2. Embed the archive
+
+## 2. Embed the archive
The next step is to embed the text of the questions.
-| - | label-coarse | -label-fine | -text | -
|---|---|---|---|
| 0 | -0 | -0 | -How did serfdom develop in and then leave Russia ? | -
| 1 | -1 | -1 | -What films featured the character Popeye Doyle ? | -
| 2 | -0 | -0 | -How can I find a list of celebrities ' real names ? | -
| 3 | -1 | -2 | -- What fowl grabs the spotlight after the Chinese Year of the Monkey ? - | -
| 4 | -2 | -3 | -What is the full form of .com ? | -
| 5 | -3 | -4 | -What contemptible scoundrel stole the cork from my lunch ? | -
| 6 | -3 | -5 | -What team did baseball 's St. Louis Browns become ? | -
| 7 | -3 | -6 | -What is the oldest profession ? | -
| 8 | -0 | -7 | -What are liver enzymes ? | -
| 9 | -3 | -4 | -Name the scar-faced bounty hunter of The Old West . | -
| + | label-coarse | +label-fine | +text | +
|---|---|---|---|
| 0 | +0 | +0 | +How did serfdom develop in and then leave Russia ? | +
| 1 | +1 | +1 | +What films featured the character Popeye Doyle ? | +
| 2 | +0 | +0 | +How can I find a list of celebrities ' real names ? | +
| 3 | +1 | +2 | +What fowl grabs the spotlight after the Chinese Year of the Monkey ? | +
| 4 | +2 | +3 | +What is the full form of .com ? | +
| 5 | +3 | +4 | +What contemptible scoundrel stole the cork from my lunch ? | +
| 6 | +3 | +5 | +What team did baseball 's St. Louis Browns become ? | +
| 7 | +3 | +6 | +What is the oldest profession ? | +
| 8 | +0 | +7 | +What are liver enzymes ? | +
| 9 | +3 | +4 | +Name the scar-faced bounty hunter of The Old West . | +
+
To get a thousand embeddings of this length should take about fifteen seconds.
-```python PYTHON
+
+```python
+# Get the embeddings
embeds = co.embed(texts=list(df['text']),
model=model_name,
input_type=input_type_embed).embeddings
```
-```python PYTHON
+
+```python
+# Check the dimensions of the embeddings
embeds = np.array(embeds)
embeds.shape
```
-```
-(1000, 4096)
-```
-## 3. Search using an index and nearest neighbor search
-
+ (1000, 4096)
+
+
+
+## 3. Search using an index and nearest neighbor search
+
Let's now use [Annoy](https://github.com/spotify/annoy) to build an index that stores the embeddings in a way that is optimized for fast search. This approach scales well to a large number of texts (other options include [Faiss](https://github.com/facebookresearch/faiss), [ScaNN](https://github.com/google-research/google-research/tree/master/scann), and [PyNNDescent](https://github.com/lmcinnes/pynndescent)).
After building the index, we can use it to retrieve the nearest neighbors either of existing questions (section 3.1), or of new questions that we embed (section 3.2).
-```python PYTHON
+
+```python
+# Create the search index, pass the size of embedding
search_index = AnnoyIndex(embeds.shape[1], 'angular')
+# Add all the vectors to the search index
for i in range(len(embeds)):
search_index.add_item(i, embeds[i])
@@ -199,105 +232,131 @@ search_index.build(10) # 10 trees
search_index.save('test.ann')
```
-```
-True
-```
-### 3.1. Find the neighbors of an example from the dataset
+
+ True
+
+
+
+### 3.1. Find the neighbors of an example from the dataset
If we're only interested in measuring the distance between the questions in the dataset (no outside queries), a simple way is to calculate the distance between every pair of embeddings we have.
-```python PYTHON
+
+```python
+# Choose an example (we'll retrieve others similar to it)
example_id = 92
+# Retrieve nearest neighbors
similar_item_ids = search_index.get_nns_by_item(example_id,10,
include_distances=True)
-results = pd.DataFrame(data={'texts': df.iloc[similar_item_ids[0]]['text'],
+# Format and print the text and distances
+results = pd.DataFrame(data={'texts': df.iloc[similar_item_ids[0]]['text'],
'distance': similar_item_ids[1]}).drop(example_id)
print(f"Question:'{df.iloc[example_id]['text']}'\nNearest neighbors:")
results
```
-```
-Question:'What are bear and bull markets ?'
-Nearest neighbors:
-```
+ Question:'What are bear and bull markets ?'
+ Nearest neighbors:
+
+
+
+
-
-
-
+
+
+
+
-### 3.2. Find the neighbors of a user query
+
+### 3.2. Find the neighbors of a user query
We're not limited to searching using existing items. If we get a query, we can embed it and find its nearest neighbors from the dataset.
-```python PYTHON
+
+```python
query = "What is the tallest mountain in the world?"
input_type_query = "search_query"
+# Get the query's embedding
query_embed = co.embed(texts=[query],
model=model_name,
input_type=input_type_query).embeddings
+# Retrieve the nearest neighbors
similar_item_ids = search_index.get_nns_by_vector(query_embed[0],10,
include_distances=True)
-query_results = pd.DataFrame(data={'texts': df.iloc[similar_item_ids[0]]['text'],
+# Format the results
+query_results = pd.DataFrame(data={'texts': df.iloc[similar_item_ids[0]]['text'],
'distance': similar_item_ids[1]})
@@ -305,94 +364,109 @@ print(f"Query:'{query}'\nNearest neighbors:")
print(query_results) # NOTE: Your results might look slightly different to ours.
```
-```
-Query:'What is the tallest mountain in the world?'
-Nearest neighbors:
-```
+ Query:'What is the tallest mountain in the world?'
+ Nearest neighbors:
+
+
+
+
| - | texts | -distance | -
|---|---|---|
| 614 | -What animals do you find in the stock market ? | -0.904278 | -
| 137 | -What are equity securities ? | -0.992819 | -
| 513 | -What do economists do ? | -1.066583 | -
| 307 | -What does NASDAQ stand for ? | -1.080738 | -
| 363 | -What does it mean `` Rupee Depreciates '' ? | -1.086724 | -
| 932 | -Why did the world enter a global depression in 1929 ? | -1.099370 | -
| 547 | -Where can stocks be traded on-line ? | -1.105368 | -
| 922 | -What is the difference between a median and a mean ? | -1.141870 | -
| 601 | -What is `` the bear of beers '' ? | -1.154140 | -
| + | texts | +distance | +
|---|---|---|
| 614 | +What animals do you find in the stock market ? | +0.904278 | +
| 137 | +What are equity securities ? | +0.992819 | +
| 513 | +What do economists do ? | +1.066583 | +
| 307 | +What does NASDAQ stand for ? | +1.080738 | +
| 363 | +What does it mean `` Rupee Depreciates '' ? | +1.086724 | +
| 932 | +Why did the world enter a global depression in 1929 ? | +1.099370 | +
| 547 | +Where can stocks be traded on-line ? | +1.105368 | +
| 922 | +What is the difference between a median and a mean ? | +1.141870 | +
| 601 | +What is `` the bear of beers '' ? | +1.154140 | +
-
-
-
+
+
+
+
-## 4. Visualizing the archive
+
+## 4. Visualizing the archive
Finally, let's plot out all the questions onto a 2D chart so you're able to visualize the semantic similarities of this dataset!
-```python PYTHON
+
+```python
#@title Plot the archive {display-mode: "form"}
-reducer = umap.UMAP(n_neighbors=20)
+# UMAP reduces the dimensions from 1024 to 2 dimensions that we can plot
+reducer = umap.UMAP(n_neighbors=20)
umap_embeds = reducer.fit_transform(embeds)
+# Prepare the data to plot and interactive visualization
+# using Altair
df_explore = pd.DataFrame(data={'text': df['text']})
df_explore['x'] = umap_embeds[:,0]
df_explore['y'] = umap_embeds[:,1]
+# Plot
chart = alt.Chart(df_explore).mark_circle(size=60).encode(
x=#'x',
alt.X('x',
@@ -410,10 +484,21 @@ chart = alt.Chart(df_explore).mark_circle(size=60).encode(
chart.interactive()
```
+
+
+
+
+
+
+
Hover over the points to read the text. Do you see some of the patterns in clustered points? Similar questions, or questions asking about similar topics?
-This concludes this introductory guide to semantic search using sentence embeddings. As you continue the path of building a search product additional considerations arise (like dealing with long texts, or finetuning to better improve the embeddings for a specific use case).
+This concludes this introductory guide to semantic search using sentence embeddings. As you continue the path of building a search product additional considerations arise (like dealing with long texts, or finetuning to better improve the embeddings for a specific use case).
+
We can’t wait to see what you start building! Share your projects or find support on [Discord](https://discord.com/invite/co-mmunity).
+
+
+
diff --git a/fern/pages/cookbooks/basic-tool-use.mdx b/fern/pages/cookbooks/basic-tool-use.mdx
index 6a92f89e5..d5559db4c 100644
--- a/fern/pages/cookbooks/basic-tool-use.mdx
+++ b/fern/pages/cookbooks/basic-tool-use.mdx
@@ -1,39 +1,60 @@
---
-title: Getting Started with Basic Tool Use
+title: Basic Tool Use
slug: /page/basic-tool-use
description: "This page describes how to work with Cohere's basic tool use functionality."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, tool use, AI agents"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-| - | texts | -distance | -
|---|---|---|
| 236 | -What is the name of the tallest mountain in the world ? | -0.447309 | -
| 670 | -What is the highest mountain in the world ? | -0.552254 | -
| 412 | -- What was the highest mountain on earth before Mount Everest was - discovered ? - | -0.801252 | -
| 907 | -- What mountain range is traversed by the highest railroad in the world - ? - | -0.929516 | -
| 435 | -What is the highest peak in Africa ? | -0.930806 | -
| 109 | -Where is the highest point in Japan ? | -0.977315 | -
| 901 | -What 's the longest river in the world ? | -1.064209 | -
| 114 | -What is the largest snake in the world ? | -1.076390 | -
| 962 | -What 's the second-largest island in the world ? | -1.088034 | -
| 27 | -What is the highest waterfall in the United States ? | -1.091145 | -
| + | texts | +distance | +
|---|---|---|
| 236 | +What is the name of the tallest mountain in the world ? | +0.447309 | +
| 670 | +What is the highest mountain in the world ? | +0.552254 | +
| 412 | +What was the highest mountain on earth before Mount Everest was discovered ? | +0.801252 | +
| 907 | +What mountain range is traversed by the highest railroad in the world ? | +0.929516 | +
| 435 | +What is the highest peak in Africa ? | +0.930806 | +
| 109 | +Where is the highest point in Japan ? | +0.977315 | +
| 901 | +What 's the longest river in the world ? | +1.064209 | +
| 114 | +What is the largest snake in the world ? | +1.076390 | +
| 962 | +What 's the second-largest island in the world ? | +1.088034 | +
| 27 | +What is the highest waterfall in the United States ? | +1.091145 | +
+
+
+Tool use allows customers to **connect their large language models to external tools like search engines, APIs, functions, databases**, etc.
+
+This allows the customer's model to unlock a richer set of behaviors by leveraging data stored in tools, taking actions through APIs, interacting with a vector database, querying a search engine, etc.
+
+This is particularly valuable for enterprise customers, since a lot of enterprise data lives in external sources.
+
+Tool Use consists of 4 steps:
- Step 1: the user configures the request to the model
-- Step 2: the model uses context to **determine which tool(s) to use and how**
-- Step 3: the tool calls are executed
-- Step 4: the model **generates a final answer with precise citations** based on the tool results
+- Step 2: the **model smartly decides which tool(s) to use and how**
+- Step 3: the tool calls are executed to mock database
+- Step 4: the **model generates a final answer with precise citations based on the tool results**
+
+
+```python
+# we'll use Cohere to do Tool Use
+# TODO: upgrade to "cohere>5"
+%pip install "cohere<5" --quiet
+```
+
+ [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m52.8/52.8 kB[0m [31m1.6 MB/s[0m eta [36m0:00:00[0m
+ [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m3.1/3.1 MB[0m [31m28.1 MB/s[0m eta [36m0:00:00[0m
+ [?25h
-```python PYTHON
+
+```python
import cohere, json
API_KEY = "..." # fill in your Cohere API key here
co = cohere.Client(API_KEY)
```
## Step 0: Create a mock database
+First, we'll define the mock data that our tools will query. This data represents sales reports and a product catalog.
-Before we can illustrate tool use, we first need to do some setup. Here, we'll define the mock data that our tools will query. This data represents sales reports and a product catalog.
-```python PYTHON
+```python
+# Mock database containing daily sales reports
sales_database = {
'2023-09-28': {
'total_sales_amount': 5000,
@@ -49,6 +70,7 @@ sales_database = {
}
}
+# Mock product catalog
product_catalog = {
'Electronics': [
{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20},
@@ -64,9 +86,12 @@ product_catalog = {
```
Now, we'll define the tools that simulate querying this database.
-For example, you could use the API of an enterprise sales platform.
+You could for example use the API of an enterprise sales platform.
+
+
+
-```python PYTHON
+```python
def query_daily_sales_report(day: str) -> dict:
"""
Function to retrieve the sales report for the given day
@@ -98,20 +123,24 @@ functions_map = {
}
```
-## Step 1 - User configures the request to the model
+## Step 1 - the user configures the request to the model
The developer provides a few things to the model:
-
- A preamble containing instructions about the task and the desired style for the output.
- The user request.
- A list of tools to the model.
- (Optionally) a chat history for the model to work with.
+
You can specify one or many tools to the model. Every tool needs to be described with a JSON schema, indicating the tool name, description, and parameters (code snippets below).
In our example, we provide two tools to the model: `daily_sales_report` and `product_catalog`.
-```python PYTHON
+
+
+```python
+# tool descriptions that the model has access to
+# note: Cohere always adds a "directly_answer" tool under the hood, so that the model can decide to not leverage any tool, if they're not needed.
tools = [
{
"name": "query_daily_sales_report",
@@ -138,29 +167,32 @@ tools = [
]
```
-Now let's define the user request.
+Now let's define the user request.
In our example we'll use: "Can you provide a sales summary for 29th September 2023, and also give me the details of all products in the 'Electronics' category that were sold that day, including their prices and stock levels?"
Only a langage model with Tool Use can answer this request: it requires looking up information in the right external tools (step 2), and then providing a final answer based on the tool results (step 4).
-```python PYTHON
+
+```python
+# preamble containing instructions about the task and the desired style for the output.
preamble = """
-## Task & Context
+## Task & Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.
"""
+# user request
message = "Can you provide a sales summary for 29th September 2023, and also give me some details about the products in the 'Electronics' category, for example their prices and stock levels?"
```
-## Step 2 – The model smartly decides which tool(s) to use and how
-
+## Step 2 – the model smartly decides which tool(s) to use and how
The model intelligently selects the right tool(s) to call -- and the right parameters for each tool call -- based on the content of the user message.
-```python PYTHON
+
+```python
response = co.chat(
message=message,
tools=tools,
@@ -168,32 +200,38 @@ response = co.chat(
model="command-r"
)
+# Note that the Cohere Chat API also exposes:
+# - stream (for streaming mode)
+# - chat_history
+# - among other parameters
+# See https://docs.cohere.com/reference/chat for details.
print("The model recommends doing the following tool calls:")
print("\n".join(str(tool_call) for tool_call in response.tool_calls))
```
-```
-The model recommends doing the following tool calls:
-cohere.ToolCall {
- name: query_daily_sales_report
- parameters: {'day': '2023-09-29'}
- generation_id: eaf955e3-623d-4796-bf51-23b07c66ed2c
-}
-cohere.ToolCall {
- name: query_product_catalog
- parameters: {'category': 'Electronics'}
- generation_id: eaf955e3-623d-4796-bf51-23b07c66ed2c
-}
-```
+ The model recommends doing the following tool calls:
+ cohere.ToolCall {
+ name: query_daily_sales_report
+ parameters: {'day': '2023-09-29'}
+ generation_id: eaf955e3-623d-4796-bf51-23b07c66ed2c
+ }
+ cohere.ToolCall {
+ name: query_product_catalog
+ parameters: {'category': 'Electronics'}
+ generation_id: eaf955e3-623d-4796-bf51-23b07c66ed2c
+ }
+
-## Step 3 – The tool calls are executed
+## Step 3 – the tool calls are executed
-You can now execute the appropriate calls, using the tool calls and tool parameters generated by the model.
+You can then execute the appropriate calls, using the tool calls and tool parameters generated by the model.
These tool calls return tool results that will be fed to the model in Step 4.
-```python PYTHON
+
+```python
tool_results = []
+# Iterate over the tool calls generated by the model
for tool_call in response.tool_calls:
# here is where you would call the tool recommended by the model, using the parameters recommended by the model
print(f"= running tool {tool_call.name}, with parameters: {tool_call.parameters}")
@@ -211,70 +249,77 @@ print("Tool results that will be fed back to the model in step 4:")
print(json.dumps(tool_results, indent=4))
```
-```
-= running tool query_daily_sales_report, with parameters: {'day': '2023-09-29'}
-== tool results: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]
-= running tool query_product_catalog, with parameters: {'category': 'Electronics'}
-== tool results: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]
-Tool results that will be fed back to the model in step 4:
-[
- {
- "call": {
- "name": "query_daily_sales_report",
- "parameters": {
- "day": "2023-09-29"
+ = running tool query_daily_sales_report, with parameters: {'day': '2023-09-29'}
+ == tool results: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]
+ = running tool query_product_catalog, with parameters: {'category': 'Electronics'}
+ == tool results: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]
+ Tool results that will be fed back to the model in step 4:
+ [
+ {
+ "call": {
+ "name": "query_daily_sales_report",
+ "parameters": {
+ "day": "2023-09-29"
+ },
+ "generation_id": "eaf955e3-623d-4796-bf51-23b07c66ed2c"
},
- "generation_id": "eaf955e3-623d-4796-bf51-23b07c66ed2c"
+ "outputs": [
+ {
+ "date": "2023-09-29",
+ "summary": "Total Sales Amount: 10000, Total Units Sold: 250"
+ }
+ ]
},
- "outputs": [
- {
- "date": "2023-09-29",
- "summary": "Total Sales Amount: 10000, Total Units Sold: 250"
- }
- ]
- },
- {
- "call": {
- "name": "query_product_catalog",
- "parameters": {
- "category": "Electronics"
+ {
+ "call": {
+ "name": "query_product_catalog",
+ "parameters": {
+ "category": "Electronics"
+ },
+ "generation_id": "eaf955e3-623d-4796-bf51-23b07c66ed2c"
},
- "generation_id": "eaf955e3-623d-4796-bf51-23b07c66ed2c"
- },
- "outputs": [
- {
- "category": "Electronics",
- "products": [
- {
- "product_id": "E1001",
- "name": "Smartphone",
- "price": 500,
- "stock_level": 20
- },
- {
- "product_id": "E1002",
- "name": "Laptop",
- "price": 1000,
- "stock_level": 15
- },
- {
- "product_id": "E1003",
- "name": "Tablet",
- "price": 300,
- "stock_level": 25
- }
- ]
- }
- ]
- }
-]
-```
+ "outputs": [
+ {
+ "category": "Electronics",
+ "products": [
+ {
+ "product_id": "E1001",
+ "name": "Smartphone",
+ "price": 500,
+ "stock_level": 20
+ },
+ {
+ "product_id": "E1002",
+ "name": "Laptop",
+ "price": 1000,
+ "stock_level": 15
+ },
+ {
+ "product_id": "E1003",
+ "name": "Tablet",
+ "price": 300,
+ "stock_level": 25
+ }
+ ]
+ }
+ ]
+ }
+ ]
+
+
+## Step 4 - the model generates a final answer based on the tool results
+Finally, the developer calls the Cohere model, providing the tools results, in order to generate the model's final answer.
+
+Bonus: At Cohere, all Tool Use calls come with... **precise citations**! 🎉
+The model cites which tool results were used to generate the final answer.
+These citations make it easy to check where the model’s generated response claims are coming from.
+They help users gain visibility into the model reasoning, and sanity check the final model generation.
+These citations are optional — you can decide to ignore them.
+
-## Step 4 - The model generates a final answer based on the tool results
-Finally, the developer calls the Cohere model, providing the tools results, in order to generate the model's final answer.
-```python PYTHON
+```python
response = co.chat(
message=message,
tools=tools,
@@ -285,57 +330,58 @@ response = co.chat(
)
```
-```python PYTHON
+
+```python
print("Final answer:")
print(response.text)
```
-```
-Final answer:
-On the 29th of September 2023, there were 10,000 sales with 250 units sold.
-
-The Electronics category contains three products. There are details below:
-
-| Product Name | Price | Stock Level |
-| ------------ | ----- | ----------- |
-| Smartphone | 500 | 20 |
-| Laptop | 1000 | 15 |
-| Tablet | 300 | 25 |
+ Final answer:
+ On the 29th of September 2023, there were 10,000 sales with 250 units sold.
+
+ The Electronics category contains three products. There are details below:
+
+ | Product Name | Price | Stock Level |
+ | ------------ | ----- | ----------- |
+ | Smartphone | 500 | 20 |
+ | Laptop | 1000 | 15 |
+ | Tablet | 300 | 25 |
+
+ The total stock level for Electronics items is 50.
-The total stock level for Electronics items is 50.
-```
## Bonus: Citations come for free with Cohere! 🎉
-At Cohere, model generations come with... precise citations! 🎉
+At Cohere, model generations come with... precise citations! 🎉
The model cites which groups of words, in the tool results, were used to generate the final answer.
These citations make it easy to check where the model’s generated response claims are coming from.
They help users gain visibility into the model reasoning, and sanity check the final model generation.
These citations are optional — you can decide to ignore them.
-```python PYTHON
+
+```python
print("Citations that support the final answer:")
for cite in response.citations:
print(cite)
```
-```
-Citations that support the final answer:
-{'start': 7, 'end': 29, 'text': '29th of September 2023', 'document_ids': ['query_daily_sales_report:0:0']}
-{'start': 42, 'end': 75, 'text': '10,000 sales with 250 units sold.', 'document_ids': ['query_daily_sales_report:0:0']}
-{'start': 112, 'end': 127, 'text': 'three products.', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 234, 'end': 244, 'text': 'Smartphone', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 247, 'end': 250, 'text': '500', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 253, 'end': 255, 'text': '20', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 260, 'end': 266, 'text': 'Laptop', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 269, 'end': 273, 'text': '1000', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 276, 'end': 278, 'text': '15', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 283, 'end': 289, 'text': 'Tablet', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 292, 'end': 295, 'text': '300', 'document_ids': ['query_product_catalog:1:0']}
-{'start': 298, 'end': 300, 'text': '25', 'document_ids': ['query_product_catalog:1:0']}
-```
+ Citations that support the final answer:
+ {'start': 7, 'end': 29, 'text': '29th of September 2023', 'document_ids': ['query_daily_sales_report:0:0']}
+ {'start': 42, 'end': 75, 'text': '10,000 sales with 250 units sold.', 'document_ids': ['query_daily_sales_report:0:0']}
+ {'start': 112, 'end': 127, 'text': 'three products.', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 234, 'end': 244, 'text': 'Smartphone', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 247, 'end': 250, 'text': '500', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 253, 'end': 255, 'text': '20', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 260, 'end': 266, 'text': 'Laptop', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 269, 'end': 273, 'text': '1000', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 276, 'end': 278, 'text': '15', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 283, 'end': 289, 'text': 'Tablet', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 292, 'end': 295, 'text': '300', 'document_ids': ['query_product_catalog:1:0']}
+ {'start': 298, 'end': 300, 'text': '25', 'document_ids': ['query_product_catalog:1:0']}
-```python PYTHON
+
+
+```python
def insert_citations_in_order(text, citations):
"""
A helper function to pretty print citations.
@@ -376,32 +422,38 @@ print(insert_citations_in_order(response.text, response.citations))
```
-```
-On the **29th of September 2023**[1], there were **10,000 sales with 250 units sold.**[1]
+ On the **29th of September 2023**[1], there were **10,000 sales with 250 units sold.**[1]
+
+ The Electronics category contains **three products.**[2] There are details below:
+
+ | Product Name | Price | Stock Level |
+ | ------------ | ----- | ----------- |
+ | **Smartphone**[2] | **500**[2] | **20**[2] |
+ | **Laptop**[2] | **1000**[2] | **15**[2] |
+ | **Tablet**[2] | **300**[2] | **25**[2] |
+
+ The total stock level for Electronics items is 50.
+
+ [1] source: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]
+ based on tool call: {'name': 'query_daily_sales_report', 'parameters': {'day': '2023-09-29'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}
+ [2] source: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]
+ based on tool call: {'name': 'query_product_catalog', 'parameters': {'category': 'Electronics'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}
-The Electronics category contains **three products.**[2] There are details below:
-| Product Name | Price | Stock Level |
-| ------------ | ----- | ----------- |
-| **Smartphone**[2] | **500**[2] | **20**[2] |
-| **Laptop**[2] | **1000**[2] | **15**[2] |
-| **Tablet**[2] | **300**[2] | **25**[2] |
+Yiha. You've used Cohere for Tool Use. Tool use opens up a wide range of new use cases. Here are a few examples:
-The total stock level for Electronics items is 50.
+- **Function calling**: It's now possible to ask the model to output a JSON object with specific function parameters.
+For instance, this allows your chatbot to interact with your CRM to change the status of a deal, or to engage with a Python interpreter to conduct data science analyses.
-[1] source: [{'date': '2023-09-29', 'summary': 'Total Sales Amount: 10000, Total Units Sold: 250'}]
- based on tool call: {'name': 'query_daily_sales_report', 'parameters': {'day': '2023-09-29'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}
-[2] source: [{'category': 'Electronics', 'products': [{'product_id': 'E1001', 'name': 'Smartphone', 'price': 500, 'stock_level': 20}, {'product_id': 'E1002', 'name': 'Laptop', 'price': 1000, 'stock_level': 15}, {'product_id': 'E1003', 'name': 'Tablet', 'price': 300, 'stock_level': 25}]}]
- based on tool call: {'name': 'query_product_catalog', 'parameters': {'category': 'Electronics'}, 'generation_id': 'eaf955e3-623d-4796-bf51-23b07c66ed2c'}
-```
+- **Query transformation**: You can transform a user message into a search query for a vector database or any search engine.
+For instance, this enables your work assistant to automatically retrieve the appropriate data from your company's documentation by creating the right query for your vector database.
-Now, you've used Cohere for Tool Use. Tool use opens up a wide range of new use cases. Here are a few examples:
+- **Advanced searches**: You can transform a user message into one-or-many queries, to do multiple subtasks based on the content of the message.
+For instance, this allows your chatbot to search across different databases and platforms to retrieve relevant information or to conduct comparative analysis.
-- **Function calling**: It's now possible to ask the model to output a JSON object with specific function parameters.
- For instance, this allows your chatbot to interact with your CRM to change the status of a deal, or to engage with a Python interpreter to conduct data science analyses.
-- **Query transformation**: You can transform a user message into a search query for a vector database or any search engine.
- For instance, this enables your work assistant to automatically retrieve the appropriate data from your company's documentation by creating the right query for your vector database.
-- **Advanced searches**: You can transform a user message into one-or-many queries, to do multiple subtasks based on the content of the message.
- For instance, this allows your chatbot to search across different databases and platforms to retrieve relevant information or to conduct comparative analysis.
+
+```python
+
+```
diff --git a/fern/pages/cookbooks/calendar-agent.mdx b/fern/pages/cookbooks/calendar-agent.mdx
index 30cb01a51..1254dd9c7 100644
--- a/fern/pages/cookbooks/calendar-agent.mdx
+++ b/fern/pages/cookbooks/calendar-agent.mdx
@@ -3,22 +3,27 @@ title: Calendar Agent with Native Multi Step Tool
slug: /page/calendar-agent
description: "This page describes how to use cohere Chat API with list_calendar_events and create_calendar_event tools to book appointments."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, AI agents"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-*
+
+
+```python
%%capture
!pip install cohere
!pip install -qU langchain-text-splitters
@@ -29,7 +33,8 @@ import { CookbookHeader } from "../../components/cookbook-header";
!pip install llama-index-postprocessor-cohere-rerank
```
-```python PYTHON
+
+```python
import requests
from typing import List
@@ -48,67 +53,99 @@ from llama_index.postprocessor.cohere_rerank import CohereRerank
from llama_index.core import VectorStoreIndex, ServiceContext
```
-```python PYTHON
+
+```python
co_model = 'command-r'
co_api_key = getpass("Enter Cohere API key: ")
co = cohere.Client(api_key=co_api_key)
```
-```txt title="Output"
-Enter Cohere API key: ··········
-```
+ Enter Cohere API key: ··········
+
-## Introduction
+
+# Introduction
Chunking is an essential component of any RAG-based system. This cookbook aims to demonstrate how different chunking strategies affect the results of LLM-generated output. There are multiple considerations that need to be taken into account when designing chunking strategy. Therefore, we begin by providing a framework for these strategies and then jump into a practical example. We will focus our example on transcript calls, which create a unique challenge because of their rich content and the change of people speaking throughout the text.
-## Chunking strategies framework [#chunking-strategies-framework]
+# Table of content
-### Document splitting
+1. [Chunking Strategies Framework](#framework)
+2. [Getting started](#getting-started)
+3. [Example 1: Chunking using content-independent strategies](#example-1)
+4. [Example 2: Chunking using content-dependent strategies](#example-2)
+5. [Discussion](#discussion)
-By document splitting, we mean deciding on the conditions under which we will break the text. At this stage, we should ask, _"Are there any parts of consecutive text we want to ensure we do not break?"_. If the answer is "no", then, the content-independent splitting strategies are helpful. On the other hand, in scenarios like transcripts or meeting notes, we probably would like to keep the content of one speaker together, which might require us to deploy content-dependent strategies.
+
+# Chunking Strategies Framework
-#### Content-independent splitting strategies
+## Document splitting
-We split the document based on some content-independent conditions, among the most popular ones are:
+By document splitting, we mean deciding on the conditions under which we will break the text. At this stage, we should ask, *"Are there any parts of consecutive text we want to ensure we do not break?"*. If the answer is "no", then, the content-independent splitting strategies are helpful. On the other hand, in scenarios like transcripts or meeting notes, we probably would like to keep the content of one speaker together, which might require us to deploy content-dependent strategies.
+### Content-independent splitting strategies
+
+We split the document based on some content-independent conditions, among the most popular ones are:
- splitting by the number of characters,
- splitting by sentence,
- splitting by a given character, for example, `\n` for paragraphs.
The advantage of this scenario is that we do not need to make any assumptions about the text. However, some considerations remain, like whether we want to preserve some semantic structure, for example, sentences or paragraphs. Sentence splitting is better suited if we are looking for small chunks to ensure accuracy. Conversely, paragraphs preserve more context and might be more useful in open-ended questions.
-#### Content-dependent splitting strategies
+### Content-dependent splitting strategies
On the other hand, there are scenarios in which we care about preserving some text structure. Then, we develop custom splitting strategies based on the document's content. A prime example is call transcripts. In such scenarios, we aim to ensure that one person's speech is fully contained within a chunk.
-### Creating chunks from the document splits
+## Creating chunks from the document splits
After the document is split, we need to decide on the desired **size** of our chunks (the split only defines how we break the document, but we can create bigger chunks from multiple splits).
Smaller chunks support more accurate retrieval. However, they might lack context. On the other hand, larger chunks offer more context, but they reduce the effectiveness of the retrieval. It is important to experiment with different settings to find the optimal balance.
-### Overlapping chunks
+## Overlapping chunks
Overlapping chunks is a useful technique to have in the toolbox. Especially when we employ content-independent splitting strategies, it helps us mitigate some of the pitfalls of breaking the document without fully understanding the text. Overlapping guarantees that there is always some buffer between the chunks, and even if an important piece of information might be split in the original splitting strategy, it is more probable that the full information will be captured in the next chunk. The disadvantage of this method is that it creates redundancy.
-## Getting started [#getting-started]
+
+# Getting started
Designing a robust chunking strategy is as much a science as an art. There are no straightforward answers; the most effective strategies often emerge through experimentation. Therefore, let's dive straight into an example to illustrate this concept.
+
+
+
+
+
+
## Utils
-```python PYTHON
+
+```python
def set_css():
display(HTML('''
-
+
'''))
get_ipython().events.register('pre_run_cell', set_css)
set_css()
```
-```python PYTHON
+
+
+
+
+
+
+
+```python
def insert_citations(text: str, citations: List[dict]):
"""
A helper function to pretty print citations.
@@ -128,7 +165,7 @@ def insert_citations(text: str, citations: List[dict]):
return text
-def build_retriever(documents, top_n=5):
+def build_retreiver(documents, top_n=5):
# Create the embedding model
embed_model = CohereEmbedding(
cohere_api_key=co_api_key,
@@ -150,11 +187,22 @@ def build_retriever(documents, top_n=5):
return retriever
```
-## Load the data
+
+
+
+
+
+
+##Load the data
In this example we will work with an 2023 Tesla earning call transcript.
-```python PYTHON
+
+```python
# Get all investement memos (19) in bvp repository
url_path = 'https://www.fool.com/earnings/call-transcripts/2024/01/24/tesla-tsla-q4-2023-earnings-call-transcript/'
response = requests.get(url_path)
@@ -169,20 +217,31 @@ text = '\n\n'.join([div.get_text() for div in target_divs])
print(text[:500])
```
-```txt title="Output"
-Length of the script: 385
-Example of processed text:
-Martin Viecha
-Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast.
+
-During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and
-```
-## Example 1: Chunking using content-independent strategies [#example-1]
-Let's begin with a simple content-independent strategy. We aim to answer the question, `Who mentions Jonathan Nolan?`. We chose this question as it is easily verifiable and it requires to identify the speaker. The answer to this question can be found in the downloaded transcript, here is the relevant passage:
+ Length of the script: 385
+
+ Example of processed text:
+ Martin Viecha
+
+ Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast.
+
+ During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and
+
+
+
+# Example 1: Chunking using content-independent strategies
+
+Let's begin with a simple content-independent strategy. We aim to answer the question, `Who mentiones Jonathan Nolan?`. We chose this question as it is easily verifiable and it requires to identify the speaker. The answer to this questions can be found in the dowloaded transcscript, here is the relevant passage:
+
```
Elon Musk -- Chief Executive Officer and Product Architect
@@ -190,16 +249,28 @@ Elon Musk -- Chief Executive Officer and Product Architect
Yeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.
```
-```python PYTHON
+
+```python
# Define the question
-question = "Who mentions Jonathan Nolan?"
+question = "Who mentiones Jonathan Nolan?"
```
+
+
+
+
+
+
In this case, we are more concerned about accuracy than a verbose answer, so we **focus on keeping the chunks small**. To ensure that the desired size is not exceeded, we will randomly split the list of characters, in our case `["\n\n", "\n", " ", ""]`.
We employ the `RecursiveCharacterTextSplitter` from [LangChain](https://python.langchain.com/docs/get_started/introduction) for this task.
-```python PYTHON
+
+```python
# Define the chunking function
def get_chunks(text, chunk_size, chunk_overlap):
text_splitter = RecursiveCharacterTextSplitter(
@@ -215,17 +286,27 @@ def get_chunks(text, chunk_size, chunk_overlap):
return documents
```
-### Experiment 1 - no overlap
+
+
+
+
+
+### Experiment 1 - no overlap
In our first experiment we define the chunk size as 500 and allow **no overlap between consecutive chunks**.
Subsequently, we implement the standard RAG pipeline. We feed the chunks into a retriever, selecting the `top_n` most pertinent to the query chunks, and supply them as context to the generation model. Throughout this pipeline, we leverage [Cohere's endpoints](https://docs.cohere.com/reference/about), specifically, `co.embed`, `co.re.rank`, and finally, `co.chat`.
-```python PYTHON
+
+```python
chunk_size = 500
chunk_overlap = 0
documents = get_chunks(text, chunk_size, chunk_overlap)
-retriever = build_retriever(documents)
+retriever = build_retreiver(documents)
source_nodes = retriever.retrieve(question)
print('Number of docuemnts: ',len(source_nodes))
@@ -241,42 +322,71 @@ response = response
print(response.text)
```
-```txt title="Output"
-Number of docuemnts: 5
-An unknown speaker mentions Jonathan Nolan in a conversation about the creators of Westworld. They mention that Jonathan Nolan and Lisa Joy Nolan are friends of theirs, and that they have invited them to visit the lab.
-```
+
+
+
+
+
+
+ Number of docuemnts: 5
+ An unknown speaker mentions Jonathan Nolan in a conversation about the creators of Westworld. They mention that Jonathan Nolan and Lisa Joy Nolan are friends of theirs, and that they have invited them to visit the lab.
+
A notable feature of [`co.chat`](https://docs.cohere.com/reference/chat) is its ability to ground the model's answer within the context. This means we can identify which chunks were used to generate the answer. Below, we show the previous output of the model together with the citation reference, where `[num]` represents the index of the chunk.
-```python PYTHON
+
+```python
print(insert_citations(response.text, response.citations))
```
-```txt title="Output"
-An unknown speaker [0] mentions Jonathan Nolan in a conversation about the creators of Westworld. [0] They mention that Jonathan Nolan and Lisa Joy Nolan [0] are friends [0] of theirs, and that they have invited them to visit the lab. [0]
-```
+
+
+
+
+
+
+ An unknown speaker [0] mentions Jonathan Nolan in a conversation about the creators of Westworld. [0] They mention that Jonathan Nolan and Lisa Joy Nolan [0] are friends [0] of theirs, and that they have invited them to visit the lab. [0]
+
Indeed, by printing the cited chunk, we can validate that the text was divided so that the generation model could not provide the correct response. Notably, the speaker's name is not included in the context, which is why the model refes to an `unknown speaker`.
-```python PYTHON
+
+```python
print(source_nodes[0])
```
-```python title="Output"
-{'text': "Yeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\n\nYeah.\n\nUnknown speaker\n\nWe're not entering Westworld anytime soon."}
-```
-### Experiment 2 - allow overlap
+
+
+
+
+ {'text': "Yeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\n\nYeah.\n\nUnknown speaker\n\nWe're not entering Westworld anytime soon."}
+
+
+### Experiment 2 - allow overlap
In the previous experiment, we discovered that the chunks were generated in a way that made it impossible to generate the correct answer. The name of the speaker was not included in the relevant chunk.
Therefore, this time to mitigate this issue, we **allow for overlap between consecutive chunks**.
-```python PYTHON
+
+```python
chunk_size = 500
chunk_overlap = 100
documents = get_chunks(text,chunk_size, chunk_overlap)
-retriever = build_retriever(documents)
+retriever = build_retreiver(documents)
source_nodes = retriever.retrieve(question)
print('Number of docuemnts: ',len(source_nodes))
@@ -292,36 +402,72 @@ response = response
print(response.text)
```
-```txt title="Output"
-Number of docuemnts: 5
-Elon Musk mentions Jonathan Nolan. Musk is the CEO and Product Architect of the lab that resembles the set of Westworld, a show created by Jonathan Nolan and Lisa Joy Nolan.
-```
+
+
+
+
+
+
+ Number of docuemnts: 5
+ Elon Musk mentions Jonathan Nolan. Musk is the CEO and Product Architect of the lab that resembles the set of Westworld, a show created by Jonathan Nolan and Lisa Joy Nolan.
+
Again, we can print the text along with the citations.
-```python PYTHON
+
+```python
print(insert_citations(response.text, response.citations))
```
-```txt title="Output"
-Elon Musk [0] mentions Jonathan Nolan. Musk is the CEO and Product Architect [0] of the lab [0] that resembles the set of Westworld [0], a show created by Jonathan Nolan [0] and Lisa Joy Nolan. [0]
-```
+
+
+
+
+
+
+ Elon Musk [0] mentions Jonathan Nolan. Musk is the CEO and Product Architect [0] of the lab [0] that resembles the set of Westworld [0], a show created by Jonathan Nolan [0] and Lisa Joy Nolan. [0]
+
And investigate the chunks which were used as context to answer the query.
-```python PYTHON
+
+
+
+```python
source_nodes[0]
```
-```python title="Output"
-{'text': "Yeah, not the best reference.\n\nElon Musk -- Chief Executive Officer and Product Architect\n\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\n\nYeah."}
-```
+
+
+
+
+
+
+
+
+
+ {'text': "Yeah, not the best reference.\n\nElon Musk -- Chief Executive Officer and Product Architect\n\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\n\nYeah."}
+
+
As we can see, by allowing overlap we managed to get the correct answer to our question.
-## Example 2: Chunking using content-dependent strategies [#example-2]
+
+# Example 2: Chunking using content-dependent strategies
-In the previous experiment, we provided an example of how using or not using overlapping can affect a model's performance, particularly in documents such as call transcripts where subjects change frequently. Ensuring that each chunk contains all relevant information is crucial. While we managed to retrieve the correct information by introducing overlapping into the chunking strategy, this might still not be the optimal approach for transcripts with longer speaker speeches.
+In the previous experiment, we provided an example of how using or not using overlapping can affect a model's performance, particularly in documents such as call transcripts where subjects change frequently. Ensuring that each chunk contains all relevant information is crucial. While we managed to retrieve the correct information by introducing overlapping into the chunking strategy, this might still not be the optimal approach for transcripts with longer speaker speeches.
Therefore, in this experiment, we will adopt a content-dependent strategy.
@@ -329,11 +475,12 @@ Our proposed approach entails segmenting the text whenever a new speaker begins
### Preprocess the text
-Firstly, let's observe that in the HTML text, each time the speaker changes, their name is enclosed within `Name
` tags, denoting the speaker's name in bold letters. +Firstly, let's observe that in the HTML text, each time the speaker changes, their name is enclosed within `Name
` tags, denoting the speaker's name in bold letters. To facilitate our text chunking process, we'll use the above observation and introduce a unique character sequence `###`, which we'll utilize as a marker for splitting the text. -```python PYTHON + +```python print('HTML text') print(target_divs[:3]) print('-------------------\n') @@ -351,19 +498,29 @@ text_custom = '\n'.join(text_custom) print(text_custom[:500]) ``` -```txt title="Output" -HTML text -[Martin Viecha
,Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast.
,During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and expectations as of today. Actual events or results could differ materially due to a number of risks and uncertainties, including those mentioned in our most recent filings with the SEC. [Operator instructions] But before we jump into Q&A, Elon has some opening remarks.
] -------------------- -### Martin Viecha -Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast. -During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions an -``` + + + + + + HTML text + [Martin Viecha
,Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast.
,During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions and expectations as of today. Actual events or results could differ materially due to a number of risks and uncertainties, including those mentioned in our most recent filings with the SEC. [Operator instructions] But before we jump into Q&A, Elon has some opening remarks.
] + ------------------- + + ### Martin Viecha + Good afternoon, everyone, and welcome to Tesla's fourth-quarter 2023 Q&A webcast. My name is Martin Viecha, VP of investor relations, and I'm joined today by Elon Musk, Vaibhav Taneja, and a number of other executives. Our Q4 results were announced at about 3 p.m. Central Time in the update that we published at the same link as this webcast. + During this call, we will discuss our business outlook and make forward-looking statements. These comments are based on our predictions an + In this approach, we prioritize splitting the text at the appropriate separator, `###.` To ensure this behavior, we'll use `CharacterTextSplitter` from [LangChain](https://python.langchain.com/docs/get_started/introduction), guaranteeing such behavior. From our analysis of the text and the fact that we aim to preserve entire speaker speeches intact, we anticipate that most of them will exceed a length of 500. Hence, we'll increase the chunk size to 1000. -```python PYTHON + +```python separator = "###" chunk_size = 1000 chunk_overlap = 0 @@ -379,7 +536,7 @@ text_splitter = CharacterTextSplitter( documents = text_splitter.create_documents([text_custom]) documents = [Document(text=doc.page_content) for doc in documents] -retriever = build_retriever(documents) +retriever = build_retreiver(documents) source_nodes = retriever.retrieve(question) print('Number of docuemnts: ',len(source_nodes)) @@ -394,41 +551,75 @@ response = response print(response.text) ``` -```txt title="Output" -WARNING:langchain_text_splitters.base:Created a chunk of size 5946, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 4092, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1782, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1392, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 2046, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1152, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1304, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1295, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 2090, which is longer than the specified 1000 -WARNING:langchain_text_splitters.base:Created a chunk of size 1251, which is longer than the specified 1000 - - -Number of docuemnts: 5 -Elon Musk mentions Jonathan Nolan. Musk is friends with the creators of Westworld, Jonathan Nolan and Lisa Joy Nolan. -``` + + + + + + + WARNING:langchain_text_splitters.base:Created a chunk of size 5946, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 4092, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1782, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1392, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 2046, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1152, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1304, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1295, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 2090, which is longer than the specified 1000 + WARNING:langchain_text_splitters.base:Created a chunk of size 1251, which is longer than the specified 1000 + + + Number of docuemnts: 5 + Elon Musk mentions Jonathan Nolan. Musk is friends with the creators of Westworld, Jonathan Nolan and Lisa Joy Nolan. + Below we validate the answer using citations. -```python PYTHON + +```python print(insert_citations(response.text, response.citations)) ``` -```txt title="Output" -Elon Musk [0] mentions Jonathan Nolan. [0] Musk is friends [0] with the creators of Westworld [0], Jonathan Nolan [0] and Lisa Joy Nolan. [0] -``` -```python PYTHON + + + + + + Elon Musk [0] mentions Jonathan Nolan. [0] Musk is friends [0] with the creators of Westworld [0], Jonathan Nolan [0] and Lisa Joy Nolan. [0] + + + +```python source_nodes[0] ``` -```python title="Output" -{'text': "Elon Musk -- Chief Executive Officer and Product Architect\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\nYeah.\n### Unknown speaker\nWe're not entering Westworld anytime soon.\n### Elon Musk -- Chief Executive Officer and Product Architect\nRight, right. Yeah. I take -- take safety very very seriously.\n### Martin Viecha\nThank you. The next question from Norman is: How many Cybertruck orders are in the queue? And when do you anticipate to be able to fulfill existing orders?"} -``` -## Discussion [#discussion] + + + + + + + + + {'text': "Elon Musk -- Chief Executive Officer and Product Architect\nYeah. The creators of Westworld, Jonathan Nolan, Lisa Joy Nolan, are friends -- are all friends of mine, actually. And I invited them to come see the lab and, like, well, come see it, hopefully soon. It's pretty well -- especially the sort of subsystem test stands where you've just got like one leg on a test stand just doing repetitive exercises and one arm on a test stand pretty well.\nYeah.\n### Unknown speaker\nWe're not entering Westworld anytime soon.\n### Elon Musk -- Chief Executive Officer and Product Architect\nRight, right. Yeah. I take -- take safety very very seriously.\n### Martin Viecha\nThank you. The next question from Norman is: How many Cybertruck orders are in the queue? And when do you anticipate to be able to fulfill existing orders?"} + + + + +# Discussion This example highlights some of the concerns that arise when implementing chunking strategies. This is a field of ongoing research, and many exciting surveys have been published in domain-specific applications. For example, this [paper](https://arxiv.org/pdf/2402.05131.pdf) examines different chunking strategies in finance. diff --git a/fern/pages/cookbooks/creating-a-qa-bot.mdx b/fern/pages/cookbooks/creating-a-qa-bot.mdx index 6c67acecb..fb50a01a2 100644 --- a/fern/pages/cookbooks/creating-a-qa-bot.mdx +++ b/fern/pages/cookbooks/creating-a-qa-bot.mdx @@ -3,34 +3,41 @@ title: Creating a QA Bot From Technical Documentation slug: /page/creating-a-qa-bot description: "This page describes how to use Cohere to build a simple question-answering system." -image: "../../assets/images/f1cc130-cohere_meta_image.jpg" +image: "../../assets/images/f1cc130-cohere_meta_image.jpg" keywords: "Cohere, AI agents, question answering systems" --- import { AuthorsContainer } from "../../components/authors-container"; import { CookbookHeader } from "../../components/cookbook-header"; -
+
+
+
+We see that retrieval works well overall: for about over 95% of questions, the golden document is consistently within the top 5 documents. However, we also notice that approx. half the false answers come from instances where the golden document wasn't retrieved (`rank = top_k = 20`). This should be improved, e.g. by adding metadata to the documents such as their section headings, or altering the chunking strategy.
There is also a non-negligible instance of false answers where the top document was retrieved. On closer inspection, many of these are due to the model phrasing its answers more verbosely than the (very laconic) golden documents. This highlights the importance of checking eval results before jumping to conclusions about model performance.
@@ -397,3 +577,10 @@ In this notebook, we've built a QA bot that answers user questions based on tech
1. How to embed the technical documentation into a vector database using Cohere embeddings and `llama_index`
2. How to build a custom retriever that leverages Cohere's `rerank`
3. How to evaluate model performance against a predetermined set of golden QA pairs
+
+
+
+
+```python
+
+```
diff --git a/fern/pages/cookbooks/csv-agent-native-api.mdx b/fern/pages/cookbooks/csv-agent-native-api.mdx
index 4d71ae851..085db8081 100644
--- a/fern/pages/cookbooks/csv-agent-native-api.mdx
+++ b/fern/pages/cookbooks/csv-agent-native-api.mdx
@@ -3,7 +3,7 @@ title: Financial CSV Agent with Native Multi-Step Cohere API
slug: /page/csv-agent-native-api
description: "This page describes how to use Cohere's models and its native API to build an agent able to work with CSV data."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, retrieval-augmented generation, RAG, AI agents, CSV"
---
@@ -14,21 +14,22 @@ import { CookbookHeader } from "../../components/cookbook-header";
authors={[
{
name: "Jason Jung",
- imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/0803e3d-Jason_Jung.jpg",
- },
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/0803e3d-Jason_Jung.jpg"
+ }
]}
/>
-
-## Motivation
+
+## Motivation
Tabular data reasoning continues to be an exciting application of Agents, particularly in the financial domain, where multiple reasoning steps may be needed to identify the right answer. For example, a financial analyst working with financial statements may be interested in computing various financial ratios with natural language queries.
Some examples may include:
-
- ROE (Return on Equity) = Net Income / Shareholder's Equity
- Net Profit Margin = Net Income / Revenue
- Asset Turnover = Revenue / Average Total Assets
@@ -38,11 +39,25 @@ Having an Agent which is able to correctly compute these and other ratios would
## Objective
-In this notebook we explore how to setup a [Cohere Agent](https://docs.cohere.com/docs/multi-step-tool-use) to answer questions over tables in Apple's SEC10K 2020 form. [Financial CSV Agent](https://docs.cohere.com/page/csv-agent) already showed how to use Langchain to ask questions about your data. This notebook will demonstrate how you can build the same agent using Cohere's native API with Langchain Python tool. We will also explore how to make your agent more resilient to errors.
+In this notebook we explore how to setup a [Cohere Agent](https://docs.cohere.com/docs/multi-step-tool-use) to answer questions over tables in Apple's SEC10K 2020 form. [financial_csv_publication.ipynb](#TODO) already showed how to use langchain to ask questions about your data. This notebook will demonstrate how you can build the same agent using Cohere's native API with langchain python tool. We will also explore how to make your agent more resilient to errors.
+
+## Table of Contents
+
+- [Setup](#setup)
+- [Define Python Tool](#define_python_tool)
+- [Create Cohere Agent](#create_cohere_agent)
+- [QnA over Single Table](#qna_over_single_table)
+- [QnA over Multiple Tables](#qna_over_multiple_tables)
+- [Error Resilience](#error_resilience)
+ - [Add Viewing Tool](#add_viewing_tool)
+
+
+
+
+# Setup
-# Setup [#setup]
-```python PYTHON
+```python
import os
from typing import List
@@ -56,12 +71,19 @@ from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_experimental.utilities import PythonREPL
```
-```python PYTHON
+
+```python
+####################################################################################################
+#
# Uncomment if you need to install the following packages
+#
+####################################################################################################
+
# !pip install --quiet langchain langchain_experimental cohere --upgrade
```
-```python PYTHON
+
+```python
# versions
print('cohere version:', cohere.__version__)
print('langchain version:', langchain.__version__)
@@ -69,195 +91,236 @@ print('langchain_core version:', langchain_core.__version__)
print('langchain_experimental version:', langchain_experimental.__version__)
```
-```txt title="Output"
-cohere version: 5.5.1
-langchain version: 0.2.0
-langchain_core version: 0.2.0
-langchain_experimental version: 0.0.59
-```
+ cohere version: 5.5.1
+ langchain version: 0.2.0
+ langchain_core version: 0.2.0
+ langchain_experimental version: 0.0.59
+
-### API Key
+### API Key
-```python PYTHON
+
+```python
COHERE_API_KEY = os.environ["COHERE_API_KEY"]
CHAT_URL= "https://api.cohere.ai/v1/chat"
COHERE_MODEL = 'command-r-plus'
co = cohere.Client(api_key=COHERE_API_KEY)
```
-### Data Loading
+### Data Loading
+
-```python PYTHON
+```python
income_statement = pd.read_csv('income_statement.csv')
balance_sheet = pd.read_csv('balance_sheet.csv')
```
-```python PYTHON
+
+```python
income_statement.head(2)
```
+
+
+
-
-
-
+
+
+
+
-```python PYTHON
+
+
+
+```python
balance_sheet.head(2)
```
+
+
+
| - | Unnamed: 0 | -index | -RevenueFromContractWithCustomerExcludingAssessedTax | -CostOfGoodsAndServicesSold | -GrossProfit | -ResearchAndDevelopmentExpense | -SellingGeneralAndAdministrativeExpense | -OperatingExpenses | -OperatingIncomeLoss | -NonoperatingIncomeExpense | -- IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest - | -IncomeTaxExpenseBenefit | -NetIncomeLoss | -EarningsPerShareBasic | -EarningsPerShareDiluted | -WeightedAverageNumberOfSharesOutstandingBasic | -WeightedAverageNumberOfDilutedSharesOutstanding | -
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -0 | -2017-10-01-2018-09-29 | -265595000000 | -1.637560e+11 | -101839000000 | -1.423600e+10 | -1.670500e+10 | -3.094100e+10 | -7.089800e+10 | -2.005000e+09 | -7.290300e+10 | -1.337200e+10 | -59531000000 | -3.00 | -2.98 | -1.982151e+10 | -2.000044e+10 | -
| 1 | -1 | -2018-09-30-2018-12-29 | -84310000000 | -NaN | -32031000000 | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -19965000000 | -1.05 | -1.05 | -NaN | -NaN | -
| + | Unnamed: 0 | +index | +RevenueFromContractWithCustomerExcludingAssessedTax | +CostOfGoodsAndServicesSold | +GrossProfit | +ResearchAndDevelopmentExpense | +SellingGeneralAndAdministrativeExpense | +OperatingExpenses | +OperatingIncomeLoss | +NonoperatingIncomeExpense | +IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest | +IncomeTaxExpenseBenefit | +NetIncomeLoss | +EarningsPerShareBasic | +EarningsPerShareDiluted | +WeightedAverageNumberOfSharesOutstandingBasic | +WeightedAverageNumberOfDilutedSharesOutstanding | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | +0 | +2017-10-01-2018-09-29 | +265595000000 | +1.637560e+11 | +101839000000 | +1.423600e+10 | +1.670500e+10 | +3.094100e+10 | +7.089800e+10 | +2.005000e+09 | +7.290300e+10 | +1.337200e+10 | +59531000000 | +3.00 | +2.98 | +1.982151e+10 | +2.000044e+10 | +
| 1 | +1 | +2018-09-30-2018-12-29 | +84310000000 | +NaN | +32031000000 | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +19965000000 | +1.05 | +1.05 | +NaN | +NaN | +
-
-
-
-
+
+
+
-# Define Python Tool [#define_python_tool]
-Here we define the python tool using langchain's PythonREPL. We also define `functions_map` that will later be used by the Cohere Agent to correctly map function name to the actual function. Lastly, we define the tools that will be passed in the Cohere API.
-```python PYTHON
+
+
+# Define Python Tool
+
+Here we define the python tool using langchain's PythonREPL. We also define `functions_map` that will later be used by the Cohere Agent to correctly map function name to the actual function. Lastly, we define the tools that will be passed in the Cohere API.
+
+
+
+```python
python_repl = PythonREPL()
python_tool = Tool(
name="python_repl",
@@ -295,11 +358,14 @@ tools = [
},]
```
-# Create Cohere Agent [#create_cohere_agent]
+
+
+# Create Cohere Agent
+
+As [Vanilla_Multi_Step_Tool_Use.ipynb](https://github.com/cohere-ai/cohere-developer-experience/blob/fbf6c8dad47d7557314e9248a267175c7a6908d8/notebooks/Vanilla_Multi_Step_Tool_Use.ipynb) shows, you have a lot of flexiblity on how you can customize and interact with the cohere agent. Here I am creating a wrapper so that it automatically determines when to stop calling the tools and output final answer. It will run maximum of 15 steps.
-As [Multi-Step Tool Use](https://docs.cohere.com/page/basic-multi-step) shows, you have a lot of flexiblity on how you can customize and interact with the cohere agent. Here I am creating a wrapper so that it automatically determines when to stop calling the tools and output final answer. It will run maximum of 15 steps.
-```python PYTHON
+```python
def cohere_agent(
message: str,
preamble: str,
@@ -373,30 +439,32 @@ def cohere_agent(
output = cohere_agent("can you use python to answer 1 + 1", None, tools, verbose=True)
```
-```txt title="Output"
-running 0th step.
-I will use Python to answer this question.
+
+ running 0th step.
+ I will use Python to answer this question.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'print(1 + 1)'}
+ == tool results: [{'python_answer': '2\n'}]
+ The answer is **2**.
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'print(1 + 1)'}
-== tool results: [{'python_answer': '2\n'}]
-The answer is **2**.
-```
-# QnA over Single Table [#qna_over_single_table]
+
+
+# QnA over Single Table
-In the example below, we show how the Python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:
+In the example below, we show how the python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:
-- Pass the file name to the preamble so the model knows how to load the dataframe
-- Pass a preview of the dataframe in the preamble so the model knows which columns/rows to query
+1) pass the file name to the preamble so the model knows how to load the dataframe
+2) pass a preview of the dataframe in the preamble so the model knows which columns/rows to query
-We will ask the following questions given income statement data.
+We will ask the following questions given income statement data.
+1. what is the highest value of cost of goods and service?
+2. what is the largest gross profit margin?
+3. what is the minimum ratio of operating income loss divided by non operating income expense?
-- What is the highest value of cost of goods and service?
-- What is the largest gross profit margin?
-- What is the minimum ratio of operating income loss divided by non operating income expense?
-```python PYTHON
+```python
question_dict ={
'q1': ['what is the highest value of cost of goods and service?',169559000000],
'q2': ['what is the largest gross profit margin?',0.3836194330595236],
@@ -404,7 +472,8 @@ question_dict ={
}
```
-```python PYTHON
+
+```python
preamble = """
You are an expert who answers the user's question. You are working with a pandas dataframe in Python. The name of the dataframe is `income_statement.csv`.
Here is a preview of the dataframe:
@@ -414,6 +483,7 @@ Here is a preview of the dataframe:
print(preamble)
```
+
You are an expert who answers the user's question. You are working with a pandas dataframe in Python. The name of the dataframe is `income_statement.csv`.
Here is a preview of the dataframe:
| | Unnamed: 0 | index | RevenueFromContractWithCustomerExcludingAssessedTax | CostOfGoodsAndServicesSold | GrossProfit | ResearchAndDevelopmentExpense | SellingGeneralAndAdministrativeExpense | OperatingExpenses | OperatingIncomeLoss | NonoperatingIncomeExpense | IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest | IncomeTaxExpenseBenefit | NetIncomeLoss | EarningsPerShareBasic | EarningsPerShareDiluted | WeightedAverageNumberOfSharesOutstandingBasic | WeightedAverageNumberOfDilutedSharesOutstanding |
@@ -421,8 +491,11 @@ print(preamble)
| 0 | 0 | 2017-10-01-2018-09-29 | 265595000000 | 1.63756e+11 | 101839000000 | 1.4236e+10 | 1.6705e+10 | 3.0941e+10 | 7.0898e+10 | 2.005e+09 | 7.2903e+10 | 1.3372e+10 | 59531000000 | 3 | 2.98 | 1.98215e+10 | 2.00004e+10 |
| 1 | 1 | 2018-09-30-2018-12-29 | 84310000000 | nan | 32031000000 | nan | nan | nan | nan | nan | nan | nan | 19965000000 | 1.05 | 1.05 | nan | nan |
| 2 | 2 | 2018-09-30-2019-09-28 | 260174000000 | 1.61782e+11 | 98392000000 | 1.6217e+10 | 1.8245e+10 | 3.4462e+10 | 6.393e+10 | 1.807e+09 | 6.5737e+10 | 1.0481e+10 | 55256000000 | 2.99 | 2.97 | 1.84713e+10 | 1.85957e+10 |
+
+
+
-```python PYTHON
+```python
for qsn,val in question_dict.items():
print(f'question:{qsn}')
question = val[0]
@@ -432,57 +505,62 @@ for qsn,val in question_dict.items():
print('-'*50)
```
-```txt title="Output"
-question:q1
-
-running 0th step.
-I will use Python to find the highest value of 'CostOfGoodsAndServicesSold' in the 'income_statement.csv' dataframe.
-
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'CostOfGoodsAndServicesSold\'\nhighest_cost = df[\'CostOfGoodsAndServicesSold\'].max()\n\nprint(f"The highest value of \'CostOfGoodsAndServicesSold\' is {highest_cost}")'}
-== tool results: [{'python_answer': "The highest value of 'CostOfGoodsAndServicesSold' is 169559000000.0\n"}]
-The highest value of 'CostOfGoodsAndServicesSold' is 169559000000.0.
-GT Answer:169559000000
---------------------------------------------------
-question:q2
-
-running 0th step.
-I will write and execute Python code to find the largest gross profit margin.
-
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Calculate gross profit margin\ndf[\'GrossProfitMargin\'] = df[\'GrossProfit\'] / df[\'RevenueFromContractWithCustomerExcludingAssessedTax\'] * 100\n\n# Find the largest gross profit margin\nlargest_gross_profit_margin = df[\'GrossProfitMargin\'].max()\n\nprint(f"The largest gross profit margin is {largest_gross_profit_margin:.2f}%")'}
-== tool results: [{'python_answer': 'The largest gross profit margin is 38.36%\n'}]
-The largest gross profit margin is 38.36%.
-GT Answer:0.3836194330595236
---------------------------------------------------
-question:q3
-
-running 0th step.
-I will use Python to find the minimum ratio of operating income loss divided by non-operating income expense.
-
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv("income_statement.csv")\n\n# Calculate the ratio of operating income loss to non-operating income expense\ndf["OperatingIncomeLossRatio"] = df["OperatingIncomeLoss"] / df["NonoperatingIncomeExpense"]\n\n# Find the minimum ratio\nmin_ratio = df["OperatingIncomeLossRatio"].min()\n\nprint(f"The minimum ratio of operating income loss to non-operating income expense is: {min_ratio:.2f}")'}
-== tool results: [{'python_answer': 'The minimum ratio of operating income loss to non-operating income expense is: 35.36\n'}]
-The minimum ratio of operating income loss to non-operating income expense is 35.36.
-GT Answer:35.360599
---------------------------------------------------
-```
+ question:q1
+
+ running 0th step.
+ I will use Python to find the highest value of 'CostOfGoodsAndServicesSold' in the 'income_statement.csv' dataframe.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'CostOfGoodsAndServicesSold\'\nhighest_cost = df[\'CostOfGoodsAndServicesSold\'].max()\n\nprint(f"The highest value of \'CostOfGoodsAndServicesSold\' is {highest_cost}")'}
+ == tool results: [{'python_answer': "The highest value of 'CostOfGoodsAndServicesSold' is 169559000000.0\n"}]
+ The highest value of 'CostOfGoodsAndServicesSold' is 169559000000.0.
+ GT Answer:169559000000
+ --------------------------------------------------
+ question:q2
+
+ running 0th step.
+ I will write and execute Python code to find the largest gross profit margin.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Calculate gross profit margin\ndf[\'GrossProfitMargin\'] = df[\'GrossProfit\'] / df[\'RevenueFromContractWithCustomerExcludingAssessedTax\'] * 100\n\n# Find the largest gross profit margin\nlargest_gross_profit_margin = df[\'GrossProfitMargin\'].max()\n\nprint(f"The largest gross profit margin is {largest_gross_profit_margin:.2f}%")'}
+ == tool results: [{'python_answer': 'The largest gross profit margin is 38.36%\n'}]
+ The largest gross profit margin is 38.36%.
+ GT Answer:0.3836194330595236
+ --------------------------------------------------
+ question:q3
+
+ running 0th step.
+ I will use Python to find the minimum ratio of operating income loss divided by non-operating income expense.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv("income_statement.csv")\n\n# Calculate the ratio of operating income loss to non-operating income expense\ndf["OperatingIncomeLossRatio"] = df["OperatingIncomeLoss"] / df["NonoperatingIncomeExpense"]\n\n# Find the minimum ratio\nmin_ratio = df["OperatingIncomeLossRatio"].min()\n\nprint(f"The minimum ratio of operating income loss to non-operating income expense is: {min_ratio:.2f}")'}
+ == tool results: [{'python_answer': 'The minimum ratio of operating income loss to non-operating income expense is: 35.36\n'}]
+ The minimum ratio of operating income loss to non-operating income expense is 35.36.
+ GT Answer:35.360599
+ --------------------------------------------------
+
+
+
+
+# QnA over Multiple Tables
+
+We now make the task for the Agent more complicated, by asking it a question that answer can be computed only by retrieving relevant information from multiple tables:
-# QnA over Multiple Tables [#qna_over_multiple_tables]
+- Q: What is the ratio of the largest stockholders equity to the smallest revenue?
-We now make the task for the Agent more complicated by asking a question that can be only answered by retrieving relevant information from multiple tables:
+As you will see below, this question can be obtained only by accessing both the balance sheet and the income statement.
-- Q: What is the ratio of the largest stockholders equity to the smallest revenue?
-As you will see below, this question can be obtained only by accessing both the balance sheet and the income statement.
-```python PYTHON
+
+```python
question_dict ={
'q1': ['what is the ratio of the largest stockholders equity to the smallest revenue'],
}
```
-```python PYTHON
+
+```python
# get the largest stockholders equity
x = balance_sheet['StockholdersEquity'].astype(float).max()
print(f"The largest stockholders equity value is: {x}")
@@ -496,13 +574,13 @@ ratio = x/y
print(f"Their ratio is: {ratio}")
```
-```txt title="Output"
-The largest stockholders equity value is: 134047000000.0
-The smallest revenue value is: 53809000000.0
-Their ratio is: 2.4911631883142227
-```
+ The largest stockholders equity value is: 134047000000.0
+ The smallest revenue value is: 53809000000.0
+ Their ratio is: 2.4911631883142227
+
-```python PYTHON
+
+```python
preamble = """
You are an expert who answers the user's question in complete sentences. You are working with two pandas dataframe in Python. Ensure your output is a string.
@@ -517,53 +595,58 @@ Here is a preview of the `balance_sheet.csv` dataframe:
print(preamble)
```
-```txt title="Output"
-You are an expert who answers the user's question in complete sentences. You are working with two pandas dataframe in Python. Ensure your output is a string.
+
+ You are an expert who answers the user's question in complete sentences. You are working with two pandas dataframe in Python. Ensure your output is a string.
+
+ Here is a preview of the `income_statement.csv` dataframe:
+ | | Unnamed: 0 | index | RevenueFromContractWithCustomerExcludingAssessedTax | CostOfGoodsAndServicesSold | GrossProfit | ResearchAndDevelopmentExpense | SellingGeneralAndAdministrativeExpense | OperatingExpenses | OperatingIncomeLoss | NonoperatingIncomeExpense | IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest | IncomeTaxExpenseBenefit | NetIncomeLoss | EarningsPerShareBasic | EarningsPerShareDiluted | WeightedAverageNumberOfSharesOutstandingBasic | WeightedAverageNumberOfDilutedSharesOutstanding |
+ |---:|-------------:|:----------------------|------------------------------------------------------:|-----------------------------:|--------------:|--------------------------------:|-----------------------------------------:|--------------------:|----------------------:|----------------------------:|----------------------------------------------------------------------------------------------:|--------------------------:|----------------:|------------------------:|--------------------------:|------------------------------------------------:|--------------------------------------------------:|
+ | 0 | 0 | 2017-10-01-2018-09-29 | 265595000000 | 1.63756e+11 | 101839000000 | 1.4236e+10 | 1.6705e+10 | 3.0941e+10 | 7.0898e+10 | 2.005e+09 | 7.2903e+10 | 1.3372e+10 | 59531000000 | 3 | 2.98 | 1.98215e+10 | 2.00004e+10 |
+ | 1 | 1 | 2018-09-30-2018-12-29 | 84310000000 | nan | 32031000000 | nan | nan | nan | nan | nan | nan | nan | 19965000000 | 1.05 | 1.05 | nan | nan |
+ | 2 | 2 | 2018-09-30-2019-09-28 | 260174000000 | 1.61782e+11 | 98392000000 | 1.6217e+10 | 1.8245e+10 | 3.4462e+10 | 6.393e+10 | 1.807e+09 | 6.5737e+10 | 1.0481e+10 | 55256000000 | 2.99 | 2.97 | 1.84713e+10 | 1.85957e+10 |
+
+ Here is a preview of the `balance_sheet.csv` dataframe:
+ | | Unnamed: 0 | index | CashAndCashEquivalentsAtCarryingValue | MarketableSecuritiesCurrent | AccountsReceivableNetCurrent | InventoryNet | NontradeReceivablesCurrent | OtherAssetsCurrent | AssetsCurrent | MarketableSecuritiesNoncurrent | PropertyPlantAndEquipmentNet | OtherAssetsNoncurrent | AssetsNoncurrent | Assets | AccountsPayableCurrent | OtherLiabilitiesCurrent | ContractWithCustomerLiabilityCurrent | CommercialPaper | LongTermDebtCurrent | LiabilitiesCurrent | LongTermDebtNoncurrent | OtherLiabilitiesNoncurrent | LiabilitiesNoncurrent | Liabilities | CommitmentsAndContingencies | CommonStocksIncludingAdditionalPaidInCapital | RetainedEarningsAccumulatedDeficit | AccumulatedOtherComprehensiveIncomeLossNetOfTax | StockholdersEquity | LiabilitiesAndStockholdersEquity |
+ |---:|-------------:|:-----------|----------------------------------------:|------------------------------:|-------------------------------:|---------------:|-----------------------------:|---------------------:|----------------:|---------------------------------:|-------------------------------:|------------------------:|-------------------:|--------------:|-------------------------:|--------------------------:|---------------------------------------:|------------------:|----------------------:|---------------------:|-------------------------:|-----------------------------:|------------------------:|--------------:|------------------------------:|-----------------------------------------------:|-------------------------------------:|--------------------------------------------------:|---------------------:|-----------------------------------:|
+ | 0 | 0 | 2017-09-30 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 134047000000 | nan |
+ | 1 | 1 | 2018-09-29 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 107147000000 | nan |
+ | 2 | 2 | 2019-09-28 | 4.8844e+10 | 5.1713e+10 | 2.2926e+10 | 4.106e+09 | 2.2878e+10 | 1.2352e+10 | 1.62819e+11 | 1.05341e+11 | 3.7378e+10 | 3.2978e+10 | 1.75697e+11 | 3.38516e+11 | 4.6236e+10 | 3.772e+10 | 5.522e+09 | 5.98e+09 | 1.026e+10 | 1.05718e+11 | 9.1807e+10 | 5.0503e+10 | 1.4231e+11 | 2.48028e+11 | 0 | 4.5174e+10 | 4.5898e+10 | -5.84e+08 | 90488000000 | 3.38516e+11 |
+
-Here is a preview of the `income_statement.csv` dataframe:
-| | Unnamed: 0 | index | RevenueFromContractWithCustomerExcludingAssessedTax | CostOfGoodsAndServicesSold | GrossProfit | ResearchAndDevelopmentExpense | SellingGeneralAndAdministrativeExpense | OperatingExpenses | OperatingIncomeLoss | NonoperatingIncomeExpense | IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest | IncomeTaxExpenseBenefit | NetIncomeLoss | EarningsPerShareBasic | EarningsPerShareDiluted | WeightedAverageNumberOfSharesOutstandingBasic | WeightedAverageNumberOfDilutedSharesOutstanding |
-|---:|-------------:|:----------------------|------------------------------------------------------:|-----------------------------:|--------------:|--------------------------------:|-----------------------------------------:|--------------------:|----------------------:|----------------------------:|----------------------------------------------------------------------------------------------:|--------------------------:|----------------:|------------------------:|--------------------------:|------------------------------------------------:|--------------------------------------------------:|
-| 0 | 0 | 2017-10-01-2018-09-29 | 265595000000 | 1.63756e+11 | 101839000000 | 1.4236e+10 | 1.6705e+10 | 3.0941e+10 | 7.0898e+10 | 2.005e+09 | 7.2903e+10 | 1.3372e+10 | 59531000000 | 3 | 2.98 | 1.98215e+10 | 2.00004e+10 |
-| 1 | 1 | 2018-09-30-2018-12-29 | 84310000000 | nan | 32031000000 | nan | nan | nan | nan | nan | nan | nan | 19965000000 | 1.05 | 1.05 | nan | nan |
-| 2 | 2 | 2018-09-30-2019-09-28 | 260174000000 | 1.61782e+11 | 98392000000 | 1.6217e+10 | 1.8245e+10 | 3.4462e+10 | 6.393e+10 | 1.807e+09 | 6.5737e+10 | 1.0481e+10 | 55256000000 | 2.99 | 2.97 | 1.84713e+10 | 1.85957e+10 |
-Here is a preview of the `balance_sheet.csv` dataframe:
-| | Unnamed: 0 | index | CashAndCashEquivalentsAtCarryingValue | MarketableSecuritiesCurrent | AccountsReceivableNetCurrent | InventoryNet | NontradeReceivablesCurrent | OtherAssetsCurrent | AssetsCurrent | MarketableSecuritiesNoncurrent | PropertyPlantAndEquipmentNet | OtherAssetsNoncurrent | AssetsNoncurrent | Assets | AccountsPayableCurrent | OtherLiabilitiesCurrent | ContractWithCustomerLiabilityCurrent | CommercialPaper | LongTermDebtCurrent | LiabilitiesCurrent | LongTermDebtNoncurrent | OtherLiabilitiesNoncurrent | LiabilitiesNoncurrent | Liabilities | CommitmentsAndContingencies | CommonStocksIncludingAdditionalPaidInCapital | RetainedEarningsAccumulatedDeficit | AccumulatedOtherComprehensiveIncomeLossNetOfTax | StockholdersEquity | LiabilitiesAndStockholdersEquity |
-|---:|-------------:|:-----------|----------------------------------------:|------------------------------:|-------------------------------:|---------------:|-----------------------------:|---------------------:|----------------:|---------------------------------:|-------------------------------:|------------------------:|-------------------:|--------------:|-------------------------:|--------------------------:|---------------------------------------:|------------------:|----------------------:|---------------------:|-------------------------:|-----------------------------:|------------------------:|--------------:|------------------------------:|-----------------------------------------------:|-------------------------------------:|--------------------------------------------------:|---------------------:|-----------------------------------:|
-| 0 | 0 | 2017-09-30 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 134047000000 | nan |
-| 1 | 1 | 2018-09-29 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 107147000000 | nan |
-| 2 | 2 | 2019-09-28 | 4.8844e+10 | 5.1713e+10 | 2.2926e+10 | 4.106e+09 | 2.2878e+10 | 1.2352e+10 | 1.62819e+11 | 1.05341e+11 | 3.7378e+10 | 3.2978e+10 | 1.75697e+11 | 3.38516e+11 | 4.6236e+10 | 3.772e+10 | 5.522e+09 | 5.98e+09 | 1.026e+10 | 1.05718e+11 | 9.1807e+10 | 5.0503e+10 | 1.4231e+11 | 2.48028e+11 | 0 | 4.5174e+10 | 4.5898e+10 | -5.84e+08 | 90488000000 | 3.38516e+11 |
-```
-```python PYTHON
+```python
for qsn,val in question_dict.items():
print(f'question:{qsn}')
question = val[0]
output = cohere_agent(question, preamble, tools, verbose=True)
```
-```txt title="Output"
-question:q1
+ question:q1
+
+ running 0th step.
+ I will use the provided code to find the ratio of the largest stockholders equity to the smallest revenue.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\n# Read the CSV files into dataframes\nincome_statement = pd.read_csv(\'income_statement.csv\')\nbalance_sheet = pd.read_csv(\'balance_sheet.csv\')\n\n# Find the smallest revenue\nsmallest_revenue = income_statement[\'RevenueFromContractWithCustomerExcludingAssessedTax\'].min()\n\n# Find the largest stockholders equity\nlargest_stockholders_equity = balance_sheet[\'StockholdersEquity\'].max()\n\n# Calculate the ratio\nratio = largest_stockholders_equity / smallest_revenue\nprint(f"The ratio of the largest stockholders equity to the smallest revenue is {ratio:.2f}")'}
+ == tool results: [{'python_answer': 'The ratio of the largest stockholders equity to the smallest revenue is 2.49\n'}]
+ The ratio of the largest stockholders equity to the smallest revenue is 2.49.
-running 0th step.
-I will use the provided code to find the ratio of the largest stockholders equity to the smallest revenue.
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\n# Read the CSV files into dataframes\nincome_statement = pd.read_csv(\'income_statement.csv\')\nbalance_sheet = pd.read_csv(\'balance_sheet.csv\')\n\n# Find the smallest revenue\nsmallest_revenue = income_statement[\'RevenueFromContractWithCustomerExcludingAssessedTax\'].min()\n\n# Find the largest stockholders equity\nlargest_stockholders_equity = balance_sheet[\'StockholdersEquity\'].max()\n\n# Calculate the ratio\nratio = largest_stockholders_equity / smallest_revenue\nprint(f"The ratio of the largest stockholders equity to the smallest revenue is {ratio:.2f}")'}
-== tool results: [{'python_answer': 'The ratio of the largest stockholders equity to the smallest revenue is 2.49\n'}]
-The ratio of the largest stockholders equity to the smallest revenue is 2.49.
-```
+
+
+# Error Resilience
-# Error Resilience [#error_resilience]
+In the previous example over single table, the model successfully answered your questions. However, the model may not always have access to the preview of the data. You will see that when we remove the preview from the preamble, the model is run into an error and not produce the answer. We will solve this problem with two different ways:
-In the previous example over single table, the model successfully answered your questions. However, the model may not always have access to the preview of the data. You will see that when we remove the preview from the preamble, the model runs into an error and is not produce the answer. We will solve this problem with two different ways:
+1. Asking the model to keep trying until it fixes the issue.
+2. Giving the model another tool to view the data and telling it to preview the data before writing code.
-- Asking the model to keep trying until it fixes the issue.
-- Giving the model another tool to view the data and telling it to preview the data before writing code.
+You will see that the second method is able to come to the answer with fewer steps.
-You will see that the second method is able to come to the answer with fewer steps.
-```python PYTHON
+
+```python
preamble = """
You are an expert who answers the user's question. You are working with a pandas dataframe in Python. The name of the dataframe is `income_statement.csv`.
"""
@@ -573,19 +656,20 @@ question1 = "what is the highest value of cost of goods and service?"
output = cohere_agent(question1, preamble, tools, verbose=True)
```
-```txt title="Output"
-running 0th step.
-I will use Python to find the highest value of 'Cost of Goods and Service' in the 'income_statement.csv' file.
+
+ running 0th step.
+ I will use Python to find the highest value of 'Cost of Goods and Service' in the 'income_statement.csv' file.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Service\'\nhighest_cost = df[\'Cost of Goods and Service\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Service\' is: {highest_cost}")'}
+ == tool results: [{'python_answer': "KeyError('Cost of Goods and Service')"}]
+ Sorry, there is no column named 'Cost of Goods and Service' in the 'income_statement.csv' file.
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Service\'\nhighest_cost = df[\'Cost of Goods and Service\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Service\' is: {highest_cost}")'}
-== tool results: [{'python_answer': "KeyError('Cost of Goods and Service')"}]
-Sorry, there is no column named 'Cost of Goods and Service' in the 'income_statement.csv' file.
-```
-As you see above, the model failed to execute because it assumed certain column names but they turned out to be wrong. One simple fix is to tell the model to continue to solve the problem in the face of error.
+As you see above, the model failed to execute because it assumed certain column names but they turned out to be wrong. One simple fix is to tell the model to continue to solve the problem in the face of error.
+
-```python PYTHON
+```python
preamble = """
You are an expert who answers the user's question. You are working with a pandas dataframe in Python. The name of the dataframe is `income_statement.csv`.
If you run into error, keep trying until you fix it. You may need to view the data to understand the error.
@@ -596,31 +680,34 @@ question1 = "what is the highest value of cost of goods and service?"
output = cohere_agent(question1, preamble, tools, verbose=True)
```
-```txt title="Output"
-running 0th step.
-I will use Python to find the highest value of cost of goods and service.
+
+ running 0th step.
+ I will use Python to find the highest value of cost of goods and service.
+
+ running 1th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Services\'\nhighest_cost = df[\'Cost of Goods and Services\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Services\' is: {highest_cost}")'}
+ == tool results: [{'python_answer': "KeyError('Cost of Goods and Services')"}]
+ I encountered an error. I will now view the data to understand the error.
+
+ running 2th step.
+ = running tool run_python_code, with parameters: {'code': "import pandas as pd\n\ndf = pd.read_csv('income_statement.csv')\n\nprint(df.columns)"}
+ == tool results: [{'python_answer': "Index(['Unnamed: 0', 'index',\n 'RevenueFromContractWithCustomerExcludingAssessedTax',\n 'CostOfGoodsAndServicesSold', 'GrossProfit',\n 'ResearchAndDevelopmentExpense',\n 'SellingGeneralAndAdministrativeExpense', 'OperatingExpenses',\n 'OperatingIncomeLoss', 'NonoperatingIncomeExpense',\n 'IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest',\n 'IncomeTaxExpenseBenefit', 'NetIncomeLoss', 'EarningsPerShareBasic',\n 'EarningsPerShareDiluted',\n 'WeightedAverageNumberOfSharesOutstandingBasic',\n 'WeightedAverageNumberOfDilutedSharesOutstanding'],\n dtype='object')\n"}]
+ I found that the column name is 'CostOfGoodsAndServicesSold'. I will now use this to find the highest value.
+
+ running 3th step.
+ = running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Services\'\nhighest_cost = df[\'CostOfGoodsAndServicesSold\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Services\' is: {highest_cost}")'}
+ == tool results: [{'python_answer': "The highest value of 'Cost of Goods and Services' is: 169559000000.0\n"}]
+ The highest value of 'Cost of Goods and Services' is 169559000000.0.
-running 1th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Services\'\nhighest_cost = df[\'Cost of Goods and Services\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Services\' is: {highest_cost}")'}
-== tool results: [{'python_answer': "KeyError('Cost of Goods and Services')"}]
-I encountered an error. I will now view the data to understand the error.
-running 2th step.
-= running tool run_python_code, with parameters: {'code': "import pandas as pd\n\ndf = pd.read_csv('income_statement.csv')\n\nprint(df.columns)"}
-== tool results: [{'python_answer': "Index(['Unnamed: 0', 'index',\n 'RevenueFromContractWithCustomerExcludingAssessedTax',\n 'CostOfGoodsAndServicesSold', 'GrossProfit',\n 'ResearchAndDevelopmentExpense',\n 'SellingGeneralAndAdministrativeExpense', 'OperatingExpenses',\n 'OperatingIncomeLoss', 'NonoperatingIncomeExpense',\n 'IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest',\n 'IncomeTaxExpenseBenefit', 'NetIncomeLoss', 'EarningsPerShareBasic',\n 'EarningsPerShareDiluted',\n 'WeightedAverageNumberOfSharesOutstandingBasic',\n 'WeightedAverageNumberOfDilutedSharesOutstanding'],\n dtype='object')\n"}]
-I found that the column name is 'CostOfGoodsAndServicesSold'. I will now use this to find the highest value.
+
-running 3th step.
-= running tool run_python_code, with parameters: {'code': 'import pandas as pd\n\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value of \'Cost of Goods and Services\'\nhighest_cost = df[\'CostOfGoodsAndServicesSold\'].max()\n\nprint(f"The highest value of \'Cost of Goods and Services\' is: {highest_cost}")'}
-== tool results: [{'python_answer': "The highest value of 'Cost of Goods and Services' is: 169559000000.0\n"}]
-The highest value of 'Cost of Goods and Services' is 169559000000.0.
-```
+### Add Viewing Tool
-### Add Viewing Tool [#add_viewing_tool]
+What if we directly give the model the ability to view the data as a tool so that it can explicitly use it instead of indirectly figuring it out?
-What if we directly give the model the ability to view the data as a tool so that it can explicitly use it instead of indirectly figuring it out?
-```python PYTHON
+```python
def view_csv_data(path: str) -> dict:
"""
Function to view the head, tail and shape of a given csv file.
@@ -664,7 +751,8 @@ tools = [
]
```
-```python PYTHON
+
+```python
preamble = """
You are an expert who answers the user's question. You are working with a pandas dataframe in Python. The name of the dataframe is `income_statement.csv`.
Always view the data first to write flawless code.
@@ -675,19 +763,19 @@ question1 = "what is the highest value of cost of goods and service?"
output = cohere_agent(question1, preamble, tools, verbose=True)
```
-```txt title="Output"
-running 0th step.
-I will first view the data and then write and execute Python code to find the highest value of cost of goods and service.
-
-running 1th step.
-= running tool view_csv_data, with parameters: {'path': 'income_statement.csv'}
-== tool results: [{'head': ' Unnamed: 0 index RevenueFromContractWithCustomerExcludingAssessedTax CostOfGoodsAndServicesSold GrossProfit ResearchAndDevelopmentExpense SellingGeneralAndAdministrativeExpense OperatingExpenses OperatingIncomeLoss NonoperatingIncomeExpense IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest IncomeTaxExpenseBenefit NetIncomeLoss EarningsPerShareBasic EarningsPerShareDiluted WeightedAverageNumberOfSharesOutstandingBasic WeightedAverageNumberOfDilutedSharesOutstanding\n0 0 2017-10-01-2018-09-29 265595000000 1.637560e+11 101839000000 1.423600e+10 1.670500e+10 3.094100e+10 7.089800e+10 2.005000e+09 7.290300e+10 1.337200e+10 59531000000 3.00 2.98 1.982151e+10 2.000044e+10\n1 1 2018-09-30-2018-12-29 84310000000 NaN 32031000000 NaN NaN NaN NaN NaN NaN NaN 19965000000 1.05 1.05 NaN NaN\n2 2 2018-09-30-2019-09-28 260174000000 1.617820e+11 98392000000 1.621700e+10 1.824500e+10 3.446200e+10 6.393000e+10 1.807000e+09 6.573700e+10 1.048100e+10 55256000000 2.99 2.97 1.847134e+10 1.859565e+10\n3 3 2018-12-30-2019-03-30 58015000000 NaN 21821000000 NaN NaN NaN NaN NaN NaN NaN 11561000000 0.62 0.61 NaN NaN\n4 4 2019-03-31-2019-06-29 53809000000 NaN 20227000000 NaN NaN NaN NaN NaN NaN NaN 10044000000 0.55 0.55 NaN NaN', 'tail': ' Unnamed: 0 index RevenueFromContractWithCustomerExcludingAssessedTax CostOfGoodsAndServicesSold GrossProfit ResearchAndDevelopmentExpense SellingGeneralAndAdministrativeExpense OperatingExpenses OperatingIncomeLoss NonoperatingIncomeExpense IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest IncomeTaxExpenseBenefit NetIncomeLoss EarningsPerShareBasic EarningsPerShareDiluted WeightedAverageNumberOfSharesOutstandingBasic WeightedAverageNumberOfDilutedSharesOutstanding\n6 6 2019-09-29-2019-12-28 91819000000 NaN 35217000000 NaN NaN NaN NaN NaN NaN NaN 22236000000 1.26 1.25 NaN NaN\n7 7 2019-09-29-2020-09-26 274515000000 1.695590e+11 104956000000 1.875200e+10 1.991600e+10 3.866800e+10 6.628800e+10 803000000.0 6.709100e+10 9.680000e+09 57411000000 3.31 3.28 1.735212e+10 1.752821e+10\n8 8 2019-12-29-2020-03-28 58313000000 NaN 22370000000 NaN NaN NaN NaN NaN NaN NaN 11249000000 0.64 0.64 NaN NaN\n9 9 2020-03-29-2020-06-27 59685000000 NaN 22680000000 NaN NaN NaN NaN NaN NaN NaN 11253000000 0.65 0.65 NaN NaN\n10 10 2020-06-28-2020-09-26 64698000000 NaN 24689000000 NaN NaN NaN NaN NaN NaN NaN 12673000000 0.74 0.73 NaN NaN', 'shape': '(11, 17)'}]
-The column name is 'CostOfGoodsAndServicesSold'. I will now write and execute Python code to find the highest value in this column.
-
-running 2th step.
-= running tool run_python_code, with parameters: {'code': "import pandas as pd\n\ndf = pd.read_csv('income_statement.csv')\n\nprint(df['CostOfGoodsAndServicesSold'].max())"}
-== tool results: [{'python_answer': '169559000000.0\n'}]
-The highest value of cost of goods and services is 169559000000.0.
-```
-
-By being prescriptive, we were able to cut down a step and get to the answer faster.
+
+ running 0th step.
+ I will first view the data and then write and execute Python code to find the highest value of cost of goods and service.
+
+ running 1th step.
+ = running tool view_csv_data, with parameters: {'path': 'income_statement.csv'}
+ == tool results: [{'head': ' Unnamed: 0 index RevenueFromContractWithCustomerExcludingAssessedTax CostOfGoodsAndServicesSold GrossProfit ResearchAndDevelopmentExpense SellingGeneralAndAdministrativeExpense OperatingExpenses OperatingIncomeLoss NonoperatingIncomeExpense IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest IncomeTaxExpenseBenefit NetIncomeLoss EarningsPerShareBasic EarningsPerShareDiluted WeightedAverageNumberOfSharesOutstandingBasic WeightedAverageNumberOfDilutedSharesOutstanding\n0 0 2017-10-01-2018-09-29 265595000000 1.637560e+11 101839000000 1.423600e+10 1.670500e+10 3.094100e+10 7.089800e+10 2.005000e+09 7.290300e+10 1.337200e+10 59531000000 3.00 2.98 1.982151e+10 2.000044e+10\n1 1 2018-09-30-2018-12-29 84310000000 NaN 32031000000 NaN NaN NaN NaN NaN NaN NaN 19965000000 1.05 1.05 NaN NaN\n2 2 2018-09-30-2019-09-28 260174000000 1.617820e+11 98392000000 1.621700e+10 1.824500e+10 3.446200e+10 6.393000e+10 1.807000e+09 6.573700e+10 1.048100e+10 55256000000 2.99 2.97 1.847134e+10 1.859565e+10\n3 3 2018-12-30-2019-03-30 58015000000 NaN 21821000000 NaN NaN NaN NaN NaN NaN NaN 11561000000 0.62 0.61 NaN NaN\n4 4 2019-03-31-2019-06-29 53809000000 NaN 20227000000 NaN NaN NaN NaN NaN NaN NaN 10044000000 0.55 0.55 NaN NaN', 'tail': ' Unnamed: 0 index RevenueFromContractWithCustomerExcludingAssessedTax CostOfGoodsAndServicesSold GrossProfit ResearchAndDevelopmentExpense SellingGeneralAndAdministrativeExpense OperatingExpenses OperatingIncomeLoss NonoperatingIncomeExpense IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest IncomeTaxExpenseBenefit NetIncomeLoss EarningsPerShareBasic EarningsPerShareDiluted WeightedAverageNumberOfSharesOutstandingBasic WeightedAverageNumberOfDilutedSharesOutstanding\n6 6 2019-09-29-2019-12-28 91819000000 NaN 35217000000 NaN NaN NaN NaN NaN NaN NaN 22236000000 1.26 1.25 NaN NaN\n7 7 2019-09-29-2020-09-26 274515000000 1.695590e+11 104956000000 1.875200e+10 1.991600e+10 3.866800e+10 6.628800e+10 803000000.0 6.709100e+10 9.680000e+09 57411000000 3.31 3.28 1.735212e+10 1.752821e+10\n8 8 2019-12-29-2020-03-28 58313000000 NaN 22370000000 NaN NaN NaN NaN NaN NaN NaN 11249000000 0.64 0.64 NaN NaN\n9 9 2020-03-29-2020-06-27 59685000000 NaN 22680000000 NaN NaN NaN NaN NaN NaN NaN 11253000000 0.65 0.65 NaN NaN\n10 10 2020-06-28-2020-09-26 64698000000 NaN 24689000000 NaN NaN NaN NaN NaN NaN NaN 12673000000 0.74 0.73 NaN NaN', 'shape': '(11, 17)'}]
+ The column name is 'CostOfGoodsAndServicesSold'. I will now write and execute Python code to find the highest value in this column.
+
+ running 2th step.
+ = running tool run_python_code, with parameters: {'code': "import pandas as pd\n\ndf = pd.read_csv('income_statement.csv')\n\nprint(df['CostOfGoodsAndServicesSold'].max())"}
+ == tool results: [{'python_answer': '169559000000.0\n'}]
+ The highest value of cost of goods and services is 169559000000.0.
+
+
+By being prescriptive, we were able to cut down a step and get to the answer faster.
diff --git a/fern/pages/cookbooks/csv-agent.mdx b/fern/pages/cookbooks/csv-agent.mdx
index d21bc7182..b89c82611 100644
--- a/fern/pages/cookbooks/csv-agent.mdx
+++ b/fern/pages/cookbooks/csv-agent.mdx
@@ -3,7 +3,7 @@ title: Financial CSV Agent with Langchain
slug: /page/csv-agent
description: "This page describes how to use Cohere's models to build an agent able to work with CSV data."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, retrieval-augmented generation, RAG, AI agents, CSV"
---
@@ -14,21 +14,21 @@ import { CookbookHeader } from "../../components/cookbook-header";
authors={[
{
name: "Shaan Desai",
- imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/d17fc44-Shaan.jpg",
- },
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/d17fc44-Shaan.jpg"
+ }
]}
/>
-| - | Unnamed: 0 | -index | -CashAndCashEquivalentsAtCarryingValue | -MarketableSecuritiesCurrent | -AccountsReceivableNetCurrent | -InventoryNet | -NontradeReceivablesCurrent | -OtherAssetsCurrent | -AssetsCurrent | -MarketableSecuritiesNoncurrent | -... | -LongTermDebtNoncurrent | -OtherLiabilitiesNoncurrent | -LiabilitiesNoncurrent | -Liabilities | -CommitmentsAndContingencies | -CommonStocksIncludingAdditionalPaidInCapital | -RetainedEarningsAccumulatedDeficit | -AccumulatedOtherComprehensiveIncomeLossNetOfTax | -StockholdersEquity | -LiabilitiesAndStockholdersEquity | -
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -0 | -2017-09-30 | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -... | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -134047000000 | -NaN | -
| 1 | -1 | -2018-09-29 | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -... | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -NaN | -107147000000 | -NaN | -
2 rows × 30 columns
+ +| + | Unnamed: 0 | +index | +CashAndCashEquivalentsAtCarryingValue | +MarketableSecuritiesCurrent | +AccountsReceivableNetCurrent | +InventoryNet | +NontradeReceivablesCurrent | +OtherAssetsCurrent | +AssetsCurrent | +MarketableSecuritiesNoncurrent | +... | +LongTermDebtNoncurrent | +OtherLiabilitiesNoncurrent | +LiabilitiesNoncurrent | +Liabilities | +CommitmentsAndContingencies | +CommonStocksIncludingAdditionalPaidInCapital | +RetainedEarningsAccumulatedDeficit | +AccumulatedOtherComprehensiveIncomeLossNetOfTax | +StockholdersEquity | +LiabilitiesAndStockholdersEquity | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | +0 | +2017-09-30 | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +... | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +134047000000 | +NaN | +
| 1 | +1 | +2018-09-29 | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +... | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +NaN | +107147000000 | +NaN | +
2 rows × 30 columns
-## Motivation
+## Motivation
Tabular data reasoning continues to be an exciting application of Agents, particularly in the financial domain, where multiple reasoning steps may be needed to identify the right answer. For example, a financial analyst working with financial statements may be interested in computing various financial ratios with natural language queries.
Some examples may include:
-
- ROE (Return on Equity) = Net Income / Shareholder's Equity
- Net Profit Margin = Net Income / Revenue
- Asset Turnover = Revenue / Average Total Assets
@@ -37,12 +37,22 @@ Some examples may include:
Having an Agent which is able to correctly compute these and other ratios would be a great help for any analyst in the field of Finance.
## Objective
+In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over tables in Apple's SEC10K 2020 form. We show how this can be done with two variants of a langchain python tool, one that requires you to pass the full path of the dataframe and another that requires the dataframe objects to be loaded in memory. While there is no major difference between the two approaches, we present both to show how to manage files that are loaded in memory vs files in storage.
-In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over tables in Apple's SEC10K 2020 form. We show how this can be done with two variants of a Langchain Python tool, one that requires you to pass the full path of the dataframe and another that requires the dataframe objects to be loaded in memory. While there is no major difference between the two approaches, we present both to show how to manage files that are loaded in memory vs files in storage.
+## Table of Contents
-# Setup [#sec_step0]
+- [Setup](#sec_step0)
+- [Introduction](#sec_step1)
+- [QnA over Single Table](#sec_step2)
+ - [Agent with Python Tool](#sec_step2_sub1)
+ - [Agent with Python Tool that takes tables as input](#sec_step2_sub2)
+- [QnA over Multiple Tables](#sec_step3)
-```python PYTHON
+
+# Setup
+
+
+```python
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain.agents import AgentExecutor
from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
@@ -54,38 +64,49 @@ from langchain_experimental.utilities import PythonREPL
from langchain_experimental.tools.python.tool import PythonAstREPLTool
```
-```python PYTHON
+
+```python
+####################################################################################################
+#
# Uncomment if you need to install the following packages
+#
+####################################################################################################
+
#!pip install --quiet langchain langchain_cohere langchain_experimental --upgrade
#!pip install sec-api
```
-# Introduction [#sec_step1]
+
+# Introduction
The aim of this notebook is to showcase how Cohere Langchain Agents can be used to answer questions over tabular data.
-This notebook assumes the input is a set of csv files extracted from Apple's SEC 10K filings.
+This notebook assumes the input is a set of csv files extracted from Apples SEC 10K filings.
### Data Loading
-We use the sec-api to download the income statement and balance sheet from the SEC 10K .htm file. Please note that the tables need to be parsed such that the index is numerical as the python code generation struggles to filter based on index. We have processed the tables and provided them for you. They can be found [here](https://github.com/cohere-ai/cohere-developer-experience/tree/main/notebooks/agents/financial-csv-agent).
+We use the sec-api to download the income statement and balance sheet from the SEC 10K .htm file. Please note that the tables need to be parsed such that the index is numerical as the python code generation struggles to filter based on index. We have processed the tables and provided them for you.
-```python PYTHON
+
+```python
income_statement = pd.read_csv('income_statement.csv')
balance_sheet = pd.read_csv('balance_sheet.csv')
```
-# QnA over Single Table{" "} [#sec_step2]
+
+# QnA over Single Table
-## Agent with Python Tool [#sec_step2_sub1]
+
+## Agent with Python Tool
In the example below, we show how the python tool can be used to load a dataframe and extract information from it. To do this successfully we need to:
-- Pass the file name to the preamble so the model knows how to load the dataframe.
-- Pass a preview of the dataframe in the preamble so the model knows which columns/rows to query.
+1) pass the file name to the preamble so the model knows how to load the dataframe
+2) pass a preview of the dataframe in the preamble so the model knows which columns/rows to query
+
+First, let's implement the ReAct agent
-First, let's implement the ReAct agent.
-```python PYTHON
+```python
# instantiate the Cohere llm
llm = ChatCohere(model="command-r", temperature=0.1,cohere_api_key="",verbose=True)
@@ -101,7 +122,7 @@ python_tool.name = "python_interpreter"
class ToolInput(BaseModel):
code: str = Field(description="Python code to execute.")
python_tool.args_schema = ToolInput
-tools=[python_tool]
+tools=[python_tool]
# define the prompt template
prompt = ChatPromptTemplate.from_template("{input}")
@@ -128,7 +149,8 @@ Here is a preview of the dataframe:
Then, we define the dictionary including the questions we want the Agent to answer, and their answer.
-```python PYTHON
+
+```python
question_dict ={
'q1': ['what is the highest value of cost of goods and service?',169559000000],
'q2': ['what is the largest gross profit margin?',0.3836194330595236],
@@ -136,9 +158,10 @@ question_dict ={
}
```
-Let's now see how the Agent answers each of the questions.
+Let's now see how the Agent answers to each of the questions
+
-```python PYTHON
+```python
for qsn,val in question_dict.items():
print(f'question:{qsn}')
agent_executor.invoke({
@@ -149,64 +172,65 @@ for qsn,val in question_dict.items():
print('-'*50)
```
-```txt title="Output"
-question:q1
-
-
-[1m> Entering new AgentExecutor chain...[0m
-[32;1m[1;3m
-I will write and execute Python code to find the highest value of Cost of Goods and Services Sold from the provided dataframe.
-{'tool_name': 'python_interpreter', 'parameters': {'code': 'import pandas as pd\n\n# Read the CSV file into a DataFrame\ndf = pd.read_csv(\'income_statement.csv\')\n\n# Find the highest value in the \'CostOfGoodsAndServicesSold\' column\nmax_value = df[\'CostOfGoodsAndServicesSold\'].max()\n\nprint(f"The highest value of Cost of Goods and Services Sold is {max_value}")'}}
-[0m[36;1m[1;3mThe highest value of Cost of Goods and Services Sold is 169559000000.0
-[0m[32;1m[1;3mRelevant Documents: 0
-Cited Documents: 0
-Answer: The highest value of Cost of Goods and Services Sold is 169559000000.0.
-Grounded answer: The highest value of Cost of Goods and Services Sold is
+
+
+
+Tool use is a method whichs allows developers to connect Cohere's Command models to external tools like search engines, APIs, databases, and other software tools. Just like how [Retrieval-Augmented Generation (RAG)](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) allows a model to use an external data source to improve factual generation, tool use is a capability that allows retrieving data from multiple sources. But it goes beyond simply retrieving information and is able to use software tools to execute code, or even create entries in a CRM system.
In this notebook, we'll see how we can use two tools to create a simple data analyst agent that is able to search the web and run code in a python interpreter. This agent uses Cohere's Command R+ mode and Langchain.
-
+
+
+# Environment
Let's start by installing the required libraries
-```python PYTHON
+
+```python
! pip install --quiet langchain langchain_cohere langchain_experimental
```
## Setup
-
We'll need a Cohere API key here. Grab your key and paste it in the next slide if you have one, or [register](https://dashboard.cohere.ai/welcome/register) and create a new API key.
-```python PYTHON
+
+```python
### LLMs
import os
-os.environ['COHERE_API_KEY'] = "
-
-The companies with the highest number of full-time employees are Microsoft with 221,000, Apple with 161,000 and NVIDIA with 29,600.
-Grounded answer: Here is a plot showing the number of full-time employees at the three US tech companies with the highest market cap in 2024:
-
+
+ The companies with the highest number of full-time employees are Microsoft with 221,000, Apple with 161,000 and NVIDIA with 29,600.
+ Grounded answer: Here is a plot showing the number of full-time employees at the three US tech companies with the highest market cap in 2024:
+ 
+
+
+
+
+```python
+# notice that the model did websearch calls, then used python to do some analysis
+# the model can even self-debug itself if the python code was wrong
```
-
+# Let's ask another question to the data analyst -- which requires a direct answer
+
-```python PYTHON
+```python
agent_executor.invoke({
"input": "Hey how are you?",
})
```
-````txt title="Output"
-[1m> Entering new AgentExecutor chain...[0m
-[32;1m[1;3mPlan: I will respond to the user's greeting.
-Action: ```json JSON
-[
- {
- "tool_name": "directly_answer",
- "parameters": {}
- }
-]
-```
-Relevant Documents: None
-Cited Documents: None
-Answer: Hey, I'm doing well, thank you for asking! How can I help you today?
-Grounded answer: Hey, I'm doing well, thank you for asking! How can I help you today?[0m
+
+
+ [1m> Entering new AgentExecutor chain...[0m
+ [32;1m[1;3mPlan: I will respond to the user's greeting.
+ Action: ```json
+ [
+ {
+ "tool_name": "directly_answer",
+ "parameters": {}
+ }
+ ]
+ ```
+ Relevant Documents: None
+ Cited Documents: None
+ Answer: Hey, I'm doing well, thank you for asking! How can I help you today?
+ Grounded answer: Hey, I'm doing well, thank you for asking! How can I help you today?[0m
+
+ [1m> Finished chain.[0m
+
+
-[1m> Finished chain.[0m
+ {'input': 'Hey how are you?',
+ 'output': "Hey, I'm doing well, thank you for asking! How can I help you today?",
+ 'citations': []}
-{'input': 'Hey how are you?',
- 'output': "Hey, I'm doing well, thank you for asking! How can I help you today?",
- 'citations': []}
-````
+This is a simple example to get you start. There [are many tools](https://python.langchain.com/docs/integrations/tools/) you can equip your agent with. Once youre comfortable with these ideas, you can also proceed to define your tools (see [Multi-step tool use in Action](https://docs.cohere.com/docs/multi-step-tool-use#multi-step-tool-use-in-action)).
-This is a simple example to get you started. There [are many tools](https://python.langchain.com/docs/integrations/tools/) you can equip your agent with. Once you're comfortable with these ideas, you can also proceed to define your tools (see [Multi-step tool use in Action](https://docs.cohere.com/docs/multi-step-tool-use#multi-step-tool-use-in-action)).
diff --git a/fern/pages/cookbooks/document-parsing-for-enterprises.mdx b/fern/pages/cookbooks/document-parsing-for-enterprises.mdx
index b3f22f0f6..f095676e8 100644
--- a/fern/pages/cookbooks/document-parsing-for-enterprises.mdx
+++ b/fern/pages/cookbooks/document-parsing-for-enterprises.mdx
@@ -3,7 +3,7 @@ title: Advanced Document Parsing For Enterprises
slug: /page/document-parsing-for-enterprises
description: "This page describes how to use Cohere's models to build a document-parsing agent."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, AI agents, document parsing"
---
@@ -14,30 +14,36 @@ import { CookbookHeader } from "../../components/cookbook-header";
authors={[
{
name: "Giannis Chatziveroglou",
- imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/73153cb-giannis.jpeg",
- },
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/73153cb-giannis.jpeg"
+ },
+ {
+ name: "Justin Lee",
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/3678fac-Justin.jpg"
+ }
]}
/>
-+
In the context of enterprise Retrieval Augmented Generation (RAG), the information is often locked in complex file types such as PDFs. These formats are made for sharing information between humans, but not so much with language models. +
+
+In this notebook, we will use a real-world pharmaceutical drug label to test out various performant approaches to parsing PDFs. This will allow us to use [Cohere's Command-R model](https://txt.cohere.com/command-r/) in a RAG setting to answer questions and asks about this label, such as "I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of" a given pharmaceutical. +
+
+
-In this notebook, we will use a real-world pharmaceutical drug label to test out various performant approaches to parsing PDFs. This will allow us to use [Cohere's Command-R model](https://cohere.com/blog/command-r/) in a RAG setting to answer questions and asks about this label, such as "I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of" a given pharmaceutical.
-
-
-
-## PDF Parsing [#top]
+
+## PDF Parsing
We will go over five proprietary as well as open source options for processing PDFs. The parsing mechanisms demonstrated in the following sections are
-
- [Google Document AI](#gcp)
- [AWS Textract](#aws)
- [Unstructured.io](#unstructured)
@@ -46,35 +52,34 @@ We will go over five proprietary as well as open source options for processing P
By way of example, we will be parsing a [21-page PDF](https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf) containing the label for a recent FDA drug approval, the beginning of which is shown below. Then, we will perform a series of basic RAG tasks with our different parsings and evaluate their performance.
-
+
## Getting Set Up
Before we dive into the technical weeds, we need to set up the notebook's runtime and filesystem environments. The code cells below do the following:
-
- Install required libraries
- Confirm that data dependencies from the GitHub repo have been downloaded. These will be under `data/document-parsing` and contain the following:
- - the PDF document that we will be working with, `fda-approved-drug.pdf` (this can also be found here: https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf)
- - precomputed parsed documents for each parsing solution. While the point of this notebook is to illustrate how this is done, we provide the parsed final results to allow readers to skip ahead to the RAG section without having to set up the required infrastructure for each solution.)
+ - the PDF document that we will be working with, `fda-approved-drug.pdf` (this can also be found here: https://www.accessdata.fda.gov/drugsatfda_docs/label/2023/215500s000lbl.pdf)
+ - precomputed parsed documents for each parsing solution. While the point of this notebook is to illustrate how this is done, we provide the parsed final results to allow readers to skip ahead to the RAG section without having to set up the required infrastructure for each solution.)
- Add utility functions needed for later sections
-```python PYTHON
+
+```python
%%capture
! sudo apt install tesseract-ocr poppler-utils
! pip install "cohere<5" fsspec hnswlib google-cloud-documentai google-cloud-storage boto3 langchain-text-splitters llama_parse pytesseract pdf2image pandas
```
-```python PYTHON
+
+```python
data_dir = "data/document-parsing"
source_filename = "example-drug-label"
extension = "pdf"
```
-```python PYTHON
+
+```python
from pathlib import Path
sources = ["gcp", "aws", "unstructured-io", "llamaparse-text", "llamaparse-markdown", "pytesseract"]
@@ -82,23 +87,24 @@ sources = ["gcp", "aws", "unstructured-io", "llamaparse-text", "llamaparse-markd
filenames = ["{}-parsed-fda-approved-drug.txt".format(source) for source in sources]
filenames.append("fda-approved-drug.pdf")
-for filename in filenames:
+for filename in filenames:
file_path = Path(f"{data_dir}/{filename}")
if file_path.is_file() == False:
print(f"File {filename} not found at {data_dir}!")
```
### Utility Functions
-
Make sure to include the notebook's utility functions in the runtime.
-```python PYTHON
+
+```python
def store_document(path: str, doc_content: str):
with open(path, 'w') as f:
f.write(doc_content)
```
-```python PYTHON
+
+```python
import json
def insert_citations_in_order(text, citations, documents):
@@ -134,7 +140,8 @@ def insert_citations_in_order(text, citations, documents):
return text_with_citations, "\n".join(citations_reference)
```
-```python PYTHON
+
+```python
def format_docs_for_chat(documents):
return [{"id": str(index), "text": x} for index, x in enumerate(documents)]
```
@@ -143,25 +150,26 @@ def format_docs_for_chat(documents):
For demonstration purposes, we have collected and saved the parsed documents from each solution in this notebook. Skip to the [next section](#document-questions) to run RAG with Command-R on the pre-fetched versions. You can find all parsed resources in detail at the link [here](https://github.com/gchatz22/temp-cohere-resources/tree/main/data).
-### Solution 1: Google Cloud Document AI [[Back to Solutions]](#top) [#gcp]
+
+### Solution 1: Google Cloud Document AI [[Back to Solutions]](#top)
Document AI helps developers create high-accuracy processors to extract, classify, and split documents.
-
+External documentation: https://cloud.google.com/document-ai #### Parsing the document -The following block can be executed in one of two ways: - -- Inside a Google Vertex AI environment - - No authentication needed -- From this notebook - - Authentication is needed - - There are pointers inside the code on which lines to uncomment in order to make this work - +The following block can be executed in one of two ways +1. Inside a Google Vertex AI environment + - No authentication is needed +2. Inside the notebook + - Authentication needed + - There are pointers inside the code on which lines to uncomment in order to achieve that +
**Note: You can skip to the next block if you want to use the pre-existing parsed version.** -```python PYTHON + +```python """ Extracted from https://cloud.google.com/document-ai/docs/samples/documentai-batch-process-document """ @@ -273,39 +281,46 @@ def batch_process_documents( # ) ``` -```python PYTHON + +```python """ Post process parsed document and store it locally. -Make sure to run this in a Google Vertex AI environment or include a credentials file. +Make sure to run in Google Vertex AI environment or include a credentials file. """ -""" -from pathlib import Path -from collections import defaultdict +# from pathlib import Path +# from collections import defaultdict -parsed_documents = [] -combined_versioned_parsed_documents = defaultdict(list) - -# Assemble versioned documents together ({"doc_name": [(0, doc_content_0), (1, doc_content_1), ...]}). -for filename, doc_content in versioned_parsed_documents: - filename, version = "-".join(filename.split("-")[:-1]), filename.split("-")[-1] - combined_versioned_parsed_documents[filename].append((version, doc_content)) - -# Sort documents by version and join the content together. -for filename, docs in combined_versioned_parsed_documents.items(): - doc_content = " ".join([x[1] for x in sorted(docs, key=lambda x: x[0])]) - parsed_documents.append((filename, doc_content)) - -# Store parsed documents in local storage. -for filename, doc_content in parsed_documents: - file_path = "{}/{}-parsed-{}.txt".format(data_dir, "gcp", source_filename) - store_document(file_path, doc_content) -""" +# parsed_documents = [] +# combined_versioned_parsed_documents = defaultdict(list) + +# # Assemble versioned documents together ({"doc_name": [(0, doc_content_0), (1, doc_content_1), ...]}). +# for filename, doc_content in versioned_parsed_documents: +# filename, version = "-".join(filename.split("-")[:-1]), filename.split("-")[-1] +# combined_versioned_parsed_documents[filename].append((version, doc_content)) + +# # Sort documents by version and join the content together. +# for filename, docs in combined_versioned_parsed_documents.items(): +# doc_content = " ".join([x[1] for x in sorted(docs, key=lambda x: x[0])]) +# parsed_documents.append((filename, doc_content)) + +# # Store parsed documents in local storage. +# for filename, doc_content in parsed_documents: +# file_path = "{}/{}-parsed-{}.txt".format(data_dir, "gcp", source_filename) +# store_document(file_path, doc_content) ``` + + + + '\nPost process parsed document and store it locally.\nMake sure to run in Google Vertex AI environment or include a credentials file.\n' + + + #### Visualize the parsed document -```python PYTHON + +```python filename = "gcp-parsed-{}.txt".format(source_filename) with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() @@ -313,13 +328,16 @@ with open("{}/{}".format(data_dir, filename), "r") as doc: print(parsed_document[:1000]) ``` -### Solution 2: AWS Textract [[Back to Solutions]](#top) [#aws] + +### Solution 2: AWS Textract [[Back to Solutions]](#top) [Amazon Textract](https://aws.amazon.com/textract/) is an OCR service offered by AWS. It can detect text, forms, tables, and more in PDFs and images. In this section, we go over how to use Textract's asynchronous API. +
+
#### Parsing the document -We assume that you are working within the AWS ecosystem (from a SageMaker notebook, EC2 instance, a Lambda function, etc.) with valid credentials. Much of the code here is from supplemental materials created by AWS and offered here: +We assume you are working within the AWS ecosystem (from a SageMaker notebook, EC2 instance, a Lambda function, ...) with valid credentials. Much of the code here is from supplemental materials created by AWS and offered here: - https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/textract - https://github.com/aws-samples/textract-paragraph-identification/tree/main @@ -333,7 +351,8 @@ At minimum, you will need access to the following AWS resources to get started: First, we bring in the `TextractWrapper` class provided in the [AWS Code Examples repository](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/python/example_code/textract/textract_wrapper.py). This class makes it simpler to interface with the Textract service. -```python PYTHON + +```python # source: https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/textract # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. @@ -616,7 +635,10 @@ class TextractWrapper: Next, we set up Textract and S3, and provide this to an instance of `TextractWrapper`. -```python PYTHON + + + +```python import boto3 textract_client = boto3.client('textract') @@ -634,7 +656,8 @@ Asynchronous calls follow the below process: 3. Once the request is complete, Textract sends out a message to the SNS topic. This can be used in conjunction with other services such as Lambda or SQS for downstream processes. 4. The parsed results can be fetched from Textract in chunks via the job ID. -```python PYTHON + +```python bucket_name = "your-bucket-name" sns_topic_arn = "your-sns-arn" # this can be found under the topic you created in the Amazon SNS dashboard sns_role_arn = "sns-role-arn" # this is an IAM role that allows Textract to interact with SNS @@ -642,7 +665,8 @@ sns_role_arn = "sns-role-arn" # this is an IAM role that allows Textract to inte file_name = "example-drug-label.pdf" ``` -```python PYTHON + +```python # kick off a text detection job. This returns a job ID. job_id = textractWrapper.start_detection_job(bucket_name=bucket_name, document_file_name=file_name, sns_topic_arn=sns_topic_arn, sns_role_arn=sns_role_arn) @@ -650,13 +674,14 @@ job_id = textractWrapper.start_detection_job(bucket_name=bucket_name, document_f Once the job completes, this will return a dictionary with the following keys: -`dict_keys(['DocumentMetadata', 'JobStatus', 'NextToken', 'Blocks', 'AnalyzeDocumentModelVersion', 'ResponseMetadata'])` +```dict_keys(['DocumentMetadata', 'JobStatus', 'NextToken', 'Blocks', 'AnalyzeDocumentModelVersion', 'ResponseMetadata'])``` This response corresponds to one chunk of information parsed by Textract. The number of chunks a document is parsed into depends on the length of the document. The two keys we are most interested in are `Blocks` and `NextToken`. `Blocks` contains all of the information that was extracted from this chunk, while `NextToken` tells us what chunk comes next, if any. Textract returns an information-rich representation of the extracted text, such as their position on the page and hierarchical relationships with other entities, all the way down to the individual word level. Since we are only interested in the raw text, we need a way to parse through all of the chunks and their `Blocks`. Lucky for us, Amazon provides some [helper functions](https://github.com/aws-samples/textract-paragraph-identification/tree/main) for this purpose, which we utilize below. -```python PYTHON + +```python def get_text_results_from_textract(job_id): response = textract_client.get_document_text_detection(JobId=job_id) collection_of_textract_responses = [] @@ -733,7 +758,8 @@ We feed in the Job ID from before into the function `get_text_results_from_textr Finally, we can concatenate the lines into one string to pass into our downstream RAG pipeline. -```python PYTHON + +```python all_text = "\n".join([line["text"] if line else "" for line in text_info_with_line_spacing]) with open(f"aws-parsed-{source_filename}.txt", "w") as f: @@ -742,7 +768,8 @@ with open(f"aws-parsed-{source_filename}.txt", "w") as f: #### Visualize the parsed document -```python PYTHON + +```python filename = "aws-parsed-{}.txt".format(source_filename) with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() @@ -750,22 +777,23 @@ with open("{}/{}".format(data_dir, filename), "r") as doc: print(parsed_document[:1000]) ``` -### Solution 3: Unstructured.io [[Back to Solutions]](#top) [#unstructured] + +### Solution 3: Unstructured.io [[Back to Solutions]](#top) Unstructured.io provides libraries with open-source components for pre-processing text documents such as PDFs, HTML and Word Documents. - +
External documentation: https://github.com/Unstructured-IO/unstructured-api #### Parsing the document The guide assumes an endpoint exists that hosts this service. The API is offered in two forms - 1. [a hosted version](https://unstructured.io/) 2. [an OSS docker image](https://github.com/Unstructured-IO/unstructured-api?tab=readme-ov-file#dizzy-instructions-for-using-the-docker-image) - +
**Note: You can skip to the next block if you want to use the pre-existing parsed version.** -```python PYTHON + +```python import os import requests @@ -793,7 +821,8 @@ parsed_document = " ".join([parsed_entry["text"] for parsed_entry in parsed_resp print("Parsed {}".format(source_filename)) ``` -```python PYTHON + +```python """ Post process parsed document and store it locally. """ @@ -804,7 +833,8 @@ store_document(file_path, parsed_document) #### Visualize the parsed document -```python PYTHON + +```python filename = "unstructured-io-parsed-{}.txt".format(source_filename) with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() @@ -812,24 +842,27 @@ with open("{}/{}".format(data_dir, filename), "r") as doc: print(parsed_document[:1000]) ``` -### Solution 4: LlamaParse [[Back to Solutions]](#top) [#llama] + -LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. +### Solution 4: LlamaParse [[Back to Solutions]](#top) +LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. +
External documentation: https://github.com/run-llama/llama_parse #### Parsing the document The following block uses the LlamaParse cloud offering. You can learn more and fetch a respective API key for the service [here](https://cloud.llamaindex.ai/parse). - +
Parsing documents with LlamaParse offers an option for two output modes both of which we will explore and compare below - - Text - Markdown +
**Note: You can skip to the next block if you want to use the pre-existing parsed version.** -```python PYTHON + +```python import os from llama_parse import LlamaParse @@ -840,7 +873,8 @@ llama_index_api_key = "{API_KEY}" input_path = "{}/{}.{}".format(data_dir, source_filename, extension) ``` -```python PYTHON + +```python # Text mode text_parser = LlamaParse( api_key=llama_index_api_key, @@ -853,7 +887,8 @@ text_parsed_document = " ".join([parsed_entry.text for parsed_entry in text_resp print("Parsed {} to text".format(source_filename)) ``` -```python PYTHON + +```python """ Post process parsed document and store it locally. """ @@ -862,7 +897,8 @@ file_path = "{}/{}-text-parsed-fda-approved-drug.txt".format(data_dir, "llamapar store_document(file_path, text_parsed_document) ``` -```python PYTHON + +```python # Markdown mode markdown_parser = LlamaParse( api_key=llama_index_api_key, @@ -875,7 +911,8 @@ markdown_parsed_document = " ".join([parsed_entry.text for parsed_entry in markd print("Parsed {} to markdown".format(source_filename)) ``` -```python PYTHON + +```python """ Post process parsed document and store it locally. """ @@ -886,45 +923,52 @@ store_document(file_path, markdown_parsed_document) #### Visualize the parsed document -```python PYTHON + +```python # Text parsing filename = "llamaparse-text-parsed-{}.txt".format(source_filename) with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() - + print(parsed_document[:1000]) ``` -```python PYTHON + +```python # Markdown parsing filename = "llamaparse-markdown-parsed-fda-approved-drug.txt" with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() - + print(parsed_document[:1000]) ``` -### Solution 5: pdf2image + pytesseract [[Back to Solutions]](#top) [#pdf2image] + + +### Solution 5: pdf2image + pytesseract [[Back to Solutions]](#top) The final parsing method we examine does not rely on cloud services, but rather relies on two libraries: `pdf2image`, and `pytesseract`. `pytesseract` lets you perform OCR locally on images, but not PDF files. So, we first convert our PDF into a set of images via `pdf2image`. #### Parsing the document -```python PYTHON + +```python from matplotlib import pyplot as plt from pdf2image import convert_from_path import pytesseract ``` -```python PYTHON + +```python # pdf2image extracts as a list of PIL.Image objects pages = convert_from_path(filename) ``` -```python PYTHON + +```python # we look at the first page as a sanity check: plt.imshow(pages[0]) @@ -934,25 +978,27 @@ plt.show() Now, we can process the image of each page with `pytesseract` and concatenate the results to get our parsed document. -```python PYTHON + +```python label_ocr_pytesseract = "".join([pytesseract.image_to_string(page) for page in pages]) ``` -```python PYTHON + +```python print(label_ocr_pytesseract[:200]) ``` -```txt title="Output" -HIGHLIGHTS OF PRESCRIBING INFORMATION + HIGHLIGHTS OF PRESCRIBING INFORMATION + + These highlights do not include all the information needed to use + IWILFIN™ safely and effectively. See full prescribing information for + IWILFIN. + + IWILFIN™ (eflor -These highlights do not include all the information needed to use -IWILFIN™ safely and effectively. See full prescribing information for -IWILFIN. -IWILFIN™ (eflor -``` -```python PYTHON +```python label_ocr_pytesseract = "".join([pytesseract.image_to_string(page) for page in pages]) with open(f"pytesseract-parsed-{source_filename}.txt", "w") as f: @@ -961,7 +1007,8 @@ with open(f"pytesseract-parsed-{source_filename}.txt", "w") as f: #### Visualize the parsed document -```python PYTHON + +```python filename = "pytesseract-parsed-{}.txt".format(source_filename) with open("{}/{}".format(data_dir, filename), "r") as doc: parsed_document = doc.read() @@ -969,10 +1016,10 @@ with open("{}/{}".format(data_dir, filename), "r") as doc: print(parsed_document[:1000]) ``` -## Document Questions [#document-questions] + +## Document Questions We can now ask a set of simple + complex questions and see how each parsing solution performs with Command-R. The questions are - - **What are the most common adverse reactions of Iwilfin?** - Task: Simple information extraction - **What is the recommended dosage of IWILFIN on body surface area between 0.5 and 0.75?** @@ -980,12 +1027,14 @@ We can now ask a set of simple + complex questions and see how each parsing solu - **I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin.** - Task: Overall document summary -```python PYTHON + +```python import cohere co = cohere.Client(api_key="{API_KEY}") ``` -```python PYTHON + +```python """ Document Questions """ @@ -1004,11 +1053,13 @@ source = "gcp" # source = "pytesseract" ``` -## Data Ingestion [#ingestion] +## Data Ingestion + In order to set up our RAG implementation, we need to separate the parsed text into chunks and load the chunks to an index. The index will allow us to retrieve relevant passages from the document for different queries. Here, we use a simple implementation of indexing using the `hnswlib` library. Note that there are many different indexing solutions that are appropriate for specific production use cases. -```python PYTHON + +```python """ Read parsed document content and chunk data """ @@ -1042,14 +1093,16 @@ documents = [c.page_content for c in chunks_] print("Source document has been broken down to {} chunks".format(len(documents))) ``` -```python PYTHON + +```python """ Embed document chunks """ document_embeddings = co.embed(texts=documents, model="embed-english-v3.0", input_type="search_document").embeddings ``` -```python PYTHON + +```python """ Create document index and add embedded chunks """ @@ -1062,15 +1115,15 @@ index.add_items(document_embeddings, list(range(len(document_embeddings)))) print("Count:", index.element_count) ``` -```txt title="Output" -Count: 115 -``` + Count: 115 + ## Retrieval In this step, we use k-nearest neighbors to fetch the most relevant documents for our query. Once the nearest neighbors are retrieved, we use Cohere's reranker to reorder the documents in the most relevant order with regards to our input search query. -```python PYTHON + +```python """ Embed search query Fetch k nearest neighbors @@ -1084,7 +1137,8 @@ neighbors = [(result[0][0][i], result[1][0][i]) for i in range(len(result[0][0]) relevant_docs = [documents[x[0]] for x in sorted(neighbors, key=lambda x: x[1])] ``` -```python PYTHON + +```python """ Rerank retrieved documents """ @@ -1095,7 +1149,8 @@ reranked_relevant_docs = format_docs_for_chat([x.document["text"] for x in reran ## Final Step: Call Command-R + RAG! -```python PYTHON + +```python """ Call the /chat endpoint with command-r """ @@ -1117,12 +1172,14 @@ print(citations_reference) Run the code cells below to make head to head comparisons of the different parsing techniques across different questions. -```python PYTHON + +```python import pandas as pd results = pd.read_csv("{}/results-table.csv".format(data_dir)) ``` -```python PYTHON + +```python question = input(""" Question 1: What are the most common adverse reactions of Iwilfin? Question 2: What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2? @@ -1145,85 +1202,87 @@ for src in ["gcp", "aws", "unstructured-io", "llamaparse-text", "llamaparse-mark print("\n") ``` -```txt title="Output" -Question 1: What are the most common adverse reactions of Iwilfin? -Question 2: What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2? -Question 3: I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin. - -Pick which question you want to see (1,2,3): 3 -Do you want to see the references as well? References are long and noisy (y/n): n - - - -| gcp | - - -Compound Name: eflornithine hydrochloride ([0], [1], [2]) (IWILFIN ([1])™) - -Indication: used to reduce the risk of relapse in adult and paediatric patients with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded at least partially to prior multiagent, multimodality therapy. ([1], [3], [4]) - -Route of Administration: IWILFIN™ tablets ([1], [3], [4]) are taken orally twice daily ([3], [4]), with doses ranging from 192 to 768 mg based on body surface area. ([3], [4]) - -Mechanism of Action: IWILFIN™ is an ornithine decarboxylase inhibitor. ([0], [2]) - - - -| aws | + + Question 1: What are the most common adverse reactions of Iwilfin? + Question 2: What is the recommended dosage of Iwilfin on body surface area between 0.5 m2 and 0.75 m2? + Question 3: I need a succinct summary of the compound name, indication, route of administration, and mechanism of action of Iwilfin. + + Pick which question you want to see (1,2,3): 3 + Do you want to see the references as well? References are long and noisy (y/n): n + + + + | gcp | + + + Compound Name: eflornithine hydrochloride ([0], [1], [2]) (IWILFIN ([1])™) + + Indication: used to reduce the risk of relapse in adult and paediatric patients with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded at least partially to prior multiagent, multimodality therapy. ([1], [3], [4]) + + Route of Administration: IWILFIN™ tablets ([1], [3], [4]) are taken orally twice daily ([3], [4]), with doses ranging from 192 to 768 mg based on body surface area. ([3], [4]) + + Mechanism of Action: IWILFIN™ is an ornithine decarboxylase inhibitor. ([0], [2]) + + + + | aws | + + + Compound Name: eflornithine ([0], [1], [2], [3]) (IWILFIN ([0])™) + + Indication: used to reduce the risk of relapse ([0], [3]) in adults ([0], [3]) and paediatric patients ([0], [3]) with high-risk neuroblastoma (HRNB) ([0], [3]) who have responded to prior therapies. ([0], [3], [4]) + + Route of Administration: Oral ([2], [4]) + + Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([1]) + + + | unstructured-io | + + + Compound Name: Iwilfin ([1], [2], [3], [4]) (eflornithine) ([0], [2], [3], [4]) + + Indication: Iwilfin is indicated to reduce the risk of relapse ([1], [3]) in adult and paediatric patients ([1], [3]) with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded to prior anti-GD2 ([1]) immunotherapy ([1], [4]) and multi-modality therapy. ([1]) + + Route of Administration: Oral ([0], [3]) + + Mechanism of Action: Iwilfin is an ornithine decarboxylase inhibitor. ([1], [2], [3], [4]) + + + | llamaparse-text | + + + Compound Name: IWILFIN ([2], [3]) (eflornithine) ([3]) + + Indication: IWILFIN is used to reduce the risk of relapse ([1], [2], [3]) in adult and paediatric patients ([1], [2], [3]) with high-risk neuroblastoma (HRNB) ([1], [2], [3]), who have responded at least partially to certain prior therapies. ([2], [3]) + + Route of Administration: IWILFIN is administered as a tablet. ([2]) + + Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([0], [1], [4]) + + + | llamaparse-markdown | + + + Compound Name: IWILFIN ([1], [2]) (eflornithine) ([1]) + + Indication: IWILFIN is indicated to reduce the risk of relapse ([1], [2]) in adult and paediatric patients ([1], [2]) with high-risk neuroblastoma (HRNB) ([1], [2]), who have responded at least partially ([1], [2], [3]) to prior anti-GD2 immunotherapy ([1], [2]) and multiagent, multimodality therapy. ([1], [2], [3]) + + Route of Administration: Oral ([0], [1], [3], [4]) + + Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([1]) + + + | pytesseract | + + + Compound Name: IWILFIN™ ([0], [2]) (eflornithine) ([0], [2]) + + Indication: IWILFIN is indicated to reduce the risk of relapse ([0], [2]) in adult and paediatric patients ([0], [2]) with high-risk neuroblastoma (HRNB) ([0], [2]), who have responded positively to prior anti-GD2 immunotherapy and multiagent, multimodality therapy. ([0], [2], [4]) + + Route of Administration: IWILFIN is administered orally ([0], [1], [3], [4]), in the form of a tablet. ([1]) + + Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([0]) + + - -Compound Name: eflornithine ([0], [1], [2], [3]) (IWILFIN ([0])™) - -Indication: used to reduce the risk of relapse ([0], [3]) in adults ([0], [3]) and paediatric patients ([0], [3]) with high-risk neuroblastoma (HRNB) ([0], [3]) who have responded to prior therapies. ([0], [3], [4]) - -Route of Administration: Oral ([2], [4]) - -Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([1]) - - -| unstructured-io | - - -Compound Name: Iwilfin ([1], [2], [3], [4]) (eflornithine) ([0], [2], [3], [4]) - -Indication: Iwilfin is indicated to reduce the risk of relapse ([1], [3]) in adult and paediatric patients ([1], [3]) with high-risk neuroblastoma (HRNB) ([1], [3]), who have responded to prior anti-GD2 ([1]) immunotherapy ([1], [4]) and multi-modality therapy. ([1]) - -Route of Administration: Oral ([0], [3]) - -Mechanism of Action: Iwilfin is an ornithine decarboxylase inhibitor. ([1], [2], [3], [4]) - - -| llamaparse-text | - - -Compound Name: IWILFIN ([2], [3]) (eflornithine) ([3]) - -Indication: IWILFIN is used to reduce the risk of relapse ([1], [2], [3]) in adult and paediatric patients ([1], [2], [3]) with high-risk neuroblastoma (HRNB) ([1], [2], [3]), who have responded at least partially to certain prior therapies. ([2], [3]) - -Route of Administration: IWILFIN is administered as a tablet. ([2]) - -Mechanism of Action: IWILFIN is an ornithine decarboxylase inhibitor. ([0], [1], [4]) - - -| llamaparse-markdown | - - -Compound Name: IWILFIN ([1], [2]) (eflornithine) ([1]) - -Indication: IWILFIN is indicated to reduce the risk of relapse ([1], [2]) in adult and paediatric patients ([1], [2]) with high-risk neuroblastoma (HRNB) ([1], [2]), who have responded at least partially ([1], [2], [3]) to prior anti-GD2 immunotherapy ([1], [2]) and multiagent, multimodality therapy. ([1], [2], [3]) - -Route of Administration: Oral ([0], [1], [3], [4]) - -Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([1]) - - -| pytesseract | - - -Compound Name: IWILFIN™ ([0], [2]) (eflornithine) ([0], [2]) - -Indication: IWILFIN is indicated to reduce the risk of relapse ([0], [2]) in adult and paediatric patients ([0], [2]) with high-risk neuroblastoma (HRNB) ([0], [2]), who have responded positively to prior anti-GD2 immunotherapy and multiagent, multimodality therapy. ([0], [2], [4]) - -Route of Administration: IWILFIN is administered orally ([0], [1], [3], [4]), in the form of a tablet. ([1]) - -Mechanism of Action: IWILFIN acts as an ornithine decarboxylase inhibitor. ([0]) -``` diff --git a/fern/pages/cookbooks/elasticsearch-and-cohere.mdx b/fern/pages/cookbooks/elasticsearch-and-cohere.mdx index c4f6bdb06..6019f1f70 100644 --- a/fern/pages/cookbooks/elasticsearch-and-cohere.mdx +++ b/fern/pages/cookbooks/elasticsearch-and-cohere.mdx @@ -3,28 +3,38 @@ title: End-to-end RAG using Elasticsearch and Cohere slug: /page/elasticsearch-and-cohere description: "This page contains a basic tutorial on how to get Cohere and ElasticSearch to work well together." -image: "../../assets/images/f1cc130-cohere_meta_image.jpg" +image: "../../assets/images/f1cc130-cohere_meta_image.jpg" keywords: "Cohere, ElasticSearch" --- import { AuthorsContainer } from "../../components/authors-container"; import { CookbookHeader } from "../../components/cookbook-header"; -
+
+
Generative models have proven extremely useful in content idea generation. But they don’t take into account user search demand and trends. In this notebook, let’s see how we can solve that by adding keyword research into the equation.
-Read the accompanying [blog post here](https://cohere.com/blog/generative-content-keyword-research/).
+Read the accompanying [blog post here](https://txt.cohere.ai/generative-content-keyword-research/).
+
-```python PYTHON
+```python
+# Install packages
! pip install cohere -q
```
-```python PYTHON
+
+```python
import cohere
import numpy as np
import pandas as pd
@@ -30,75 +40,139 @@ import cohere
co = cohere.Client("COHERE_API_KEY") # Get your API key: https://dashboard.cohere.com/api-keys
```
-```python PYTHON
+
+
+
+
+
+
+
+```python
#@title Enable text wrapping in Google Colab
from IPython.display import HTML, display
def set_css():
display(HTML('''
-
+
'''))
get_ipython().events.register('pre_run_cell', set_css)
```
+# Step 1: Get a list of High-performing Keywords
+
First, we need to get a supply of high-traffic keywords for a given topic. We can get this via keyword research tools, of which are many available. We’ll use Google Keyword Planner, which is free to use.
-```python PYTHON
+
+```python
+# Download the pre-created dataset (feel free to replace with your CSV file, containing two columns - "keyword" and "volume")
import wget
wget.download("https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/notebooks/data/remote_teams.csv", "remote_teams.csv")
```
-```
-'remote_teams.csv'
-```
-```python PYTHON
+
+
+
+
+
+
+
+
+ 'remote_teams.csv'
+
+
+
+
+```python
+# Create a dataframe
df = pd.read_csv('remote_teams.csv')
df.columns = ["keyword","volume"]
df.head()
```
+
+
+
+
+
+
+
+
+
-
-
-
+
+
+
+
+
+
+# Step 2: Group the Keywords into Topics
+
We now have a list of keywords, but this list is still raw. For example, “managing remote teams” is the top-ranking keyword in this list. But at the same time, there are many similar keywords further down in the list, such as “how to effectively manage remote teams.”
We can do that by clustering them into topics. For this, we’ll leverage Cohere’s Embed endpoint and scikit-learn.
@@ -107,7 +181,8 @@ We can do that by clustering them into topics. For this, we’ll leverage Cohere
The Cohere Embed endpoint turns a text input into a text embedding.
-```python PYTHON
+
+```python
def embed_text(texts):
output = co.embed(
texts=texts,
@@ -119,173 +194,285 @@ def embed_text(texts):
embeds = np.array(embed_text(df['keyword'].tolist()))
```
+
+
+
+
+
+
### Cluster the Keywords into Topics with scikit-learn
We then use these embeddings to cluster the keywords. A common term used for this exercise is “topic modeling.” Here, we can leverage scikit-learn’s KMeans module, a machine learning algorithm for clustering.
-```python PYTHON
+
+```python
NUM_TOPICS = 4
kmeans = KMeans(n_clusters=NUM_TOPICS, random_state=21, n_init="auto").fit(embeds)
df['topic'] = list(kmeans.labels_)
df.head()
```
+
+
+
+
+
+
+
+
+
| - | keyword | -volume | -
|---|---|---|
| 0 | -managing remote teams | -1000 | -
| 1 | -remote teams | -390 | -
| 2 | -collaboration tools for remote teams | -320 | -
| 3 | -online games for remote teams | -320 | -
| 4 | -how to manage remote teams | -260 | -
| + | keyword | +volume | +
|---|---|---|
| 0 | +managing remote teams | +1000 | +
| 1 | +remote teams | +390 | +
| 2 | +collaboration tools for remote teams | +320 | +
| 3 | +online games for remote teams | +320 | +
| 4 | +how to manage remote teams | +260 | +
-
-
-
+
+
+
+
+
+
### Generate Topic Names with Cohere Chat
We use the Chat to generate a topic name for that cluster.
-```python PYTHON
+
+```python
+# Group the DataFrame by 'topic' and aggregate the 'keyword' column into sets (which automatically removes duplicates)
topic_keywords_dict = {topic: list(set(group['keyword'])) for topic, group in df.groupby('topic')}
```
-```python PYTHON
+
+
+
+
+
+
+
+```python
+# Function to generate a topic name based on keywords
def generate_topic_name(keywords):
# Construct the prompt
prompt = f"""Generate a concise topic name that best represents these keywords.\
Provide just the topic name and not any additional details.
Keywords: {', '.join(keywords)}"""
-
+
# Call the Cohere API
response = co.chat(
model='command-r', # Choose the model size
message=prompt,
preamble="")
-
+
# Return the generated text
return response.text
```
-```python PYTHON
+
+
+
+
+
+
+
+```python
+# Generate topic names and create a mapping of topic number to topic name
topic_name_mapping = {topic: generate_topic_name(keywords) for topic, keywords in topic_keywords_dict.items()}
+# Use the mapping to create a new column in the DataFrame
df['topic_name'] = df['topic'].map(topic_name_mapping)
+# Display the first few rows to verify the new column
df.head()
```
+
+
+
+
+
+
+
+
+
| - | keyword | -volume | -topic | -
|---|---|---|---|
| 0 | -managing remote teams | -1000 | -0 | -
| 1 | -remote teams | -390 | -1 | -
| 2 | -collaboration tools for remote teams | -320 | -1 | -
| 3 | -online games for remote teams | -320 | -3 | -
| 4 | -how to manage remote teams | -260 | -0 | -
| + | keyword | +volume | +topic | +
|---|---|---|---|
| 0 | +managing remote teams | +1000 | +0 | +
| 1 | +remote teams | +390 | +1 | +
| 2 | +collaboration tools for remote teams | +320 | +1 | +
| 3 | +online games for remote teams | +320 | +3 | +
| 4 | +how to manage remote teams | +260 | +0 | +
-
-
-
+
+
+
+
-```python PYTHON
+
+
+
+```python
+# View the list of topics
for topic, name in topic_name_mapping.items():
print(f"Topic {topic}: {name}")
```
-```
-Topic 0: Remote Team Management
-Topic 1: Remote Team Tools and Tips
-Topic 2: Remote Team Resources
-Topic 3: Remote Team Fun
-```
+
+
+
+
+
+
+ Topic 0: Remote Team Management
+ Topic 1: Remote Team Tools and Tips
+ Topic 2: Remote Team Resources
+ Topic 3: Remote Team Fun
+
+
+# Step 3: Generate Blog Post Ideas for Each Topic
Now that we have the keywords nicely grouped into topics, we can proceed to generate the content ideas.
+
### Take the Top Keywords from Each Topic
Here we can implement a filter to take just the top N keywords from each topic, sorted by the search volume. In our case, we use 10.
-```python PYTHON
+
+```python
TOP_N = 10
+# Group the DataFrame by topic and select the top N keywords sorted by volume
top_keywords = (df.groupby('topic')
.apply(lambda x: x.nlargest(TOP_N, 'volume'))
.reset_index(drop=True))
+# Convert the DataFrame to a nested dictionary
content_by_topic = {}
for topic, group in top_keywords.groupby('topic'):
keywords = ', '.join(list(group['keyword']))
@@ -294,34 +481,60 @@ for topic, group in top_keywords.groupby('topic'):
content_by_topic[topic] = {'topic_name': topic_name, 'keywords': keywords}
```
-```python PYTHON
+
+
+
+
+
+
+
+```python
+# Print the topics and they top keywords
content_by_topic
```
-```
-{0: {'topic_name': 'Remote Team Management',
- 'keywords': 'managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training'},
- 1: {'topic_name': 'Remote Team Tools and Tips',
- 'keywords': 'remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams'},
- 2: {'topic_name': 'Remote Team Resources',
- 'keywords': 'best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams'},
- 3: {'topic_name': 'Remote Team Fun',
- 'keywords': 'online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams'}}
-```
+
+
+
+
+
+
+
+
+
+ {0: {'topic_name': 'Remote Team Management',
+ 'keywords': 'managing remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, best practices for managing remote teams, manage remote teams, building culture in remote teams, culture building for remote teams, managing remote teams training'},
+ 1: {'topic_name': 'Remote Team Tools and Tips',
+ 'keywords': 'remote teams, collaboration tools for remote teams, team building for remote teams, scrum remote teams, tools for remote teams, zapier remote teams, working agreements for remote teams, working with remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams'},
+ 2: {'topic_name': 'Remote Team Resources',
+ 'keywords': 'best collaboration tools for remote teams, slack best practices for remote teams, best communication tools for remote teams, best tools for remote teams, always on video for remote teams, best apps for remote teams, best free collaboration tools for remote teams, best games for remote teams, best gifts for remote teams, best ice breaker questions for remote teams'},
+ 3: {'topic_name': 'Remote Team Fun',
+ 'keywords': 'online games for remote teams, team building activities for remote teams, games for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, retro ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams'}}
+
+
### Create a Prompt with These Keywords
Next, we use the Chat endpoint to produce the content ideas. The prompt we’ll use is as follows
-```python PYTHON
+
+```python
def generate_blog_ideas(keywords):
- prompt = f"""{keywords}\n\nThe above is a list of high-traffic keywords obtained from a keyword research tool.
-Suggest three blog post ideas that are highly relevant to these keywords.
-For each idea, write a one paragraph abstract about the topic.
+ prompt = f"""{keywords}\n\nThe above is a list of high-traffic keywords obtained from a keyword research tool.
+Suggest three blog post ideas that are highly relevant to these keywords.
+For each idea, write a one paragraph abstract about the topic.
Use this format:
Blog title: | - | keyword | -volume | -topic | -topic_name | -
|---|---|---|---|---|
| 0 | -managing remote teams | -1000 | -0 | -Remote Team Management | -
| 1 | -remote teams | -390 | -1 | -Remote Team Tools and Tips | -
| 2 | -collaboration tools for remote teams | -320 | -1 | -Remote Team Tools and Tips | -
| 3 | -online games for remote teams | -320 | -3 | -Remote Team Fun | -
| 4 | -how to manage remote teams | -260 | -0 | -Remote Team Management | -
| + | keyword | +volume | +topic | +topic_name | +
|---|---|---|---|---|
| 0 | +managing remote teams | +1000 | +0 | +Remote Team Management | +
| 1 | +remote teams | +390 | +1 | +Remote Team Tools and Tips | +
| 2 | +collaboration tools for remote teams | +320 | +1 | +Remote Team Tools and Tips | +
| 3 | +online games for remote teams | +320 | +3 | +Remote Team Fun | +
| 4 | +how to manage remote teams | +260 | +0 | +Remote Team Management | +
+
+
Here we take a quick tour of what’s possible with language AI via Cohere’s Large Language Model (LLM) API. This is the Hello, World! of language AI, written for developers with little or no background in AI. In fact, we’ll do that by exploring the Hello, World! phrase itself.
-Read the accompanying [blog post here](https://cohere.com/blog/hello-world-p1/).
+Read the accompanying [blog post here](https://txt.cohere.ai/hello-world-p1/).
-
+
We’ll cover three groups of tasks that you will typically work on when dealing with language data, including:
-
- Generating text
- Classifying text
- Analyzing text
-The first step is to install the Cohere Python SDK. Next, create an API key, which you can generate from the Cohere [dashboard](https://dashboard.cohere.com/register) or [CLI tool](https://docs.cohere.ai/cli-key).
-```python PYTHON
+The first step is to install the Cohere Python SDK. Next, create an API key, which you can generate from the Cohere [dashboard](https://os.cohere.ai/register) or [CLI tool](https://docs.cohere.ai/cli-key).
+
+
+```python
+# Install the libraries
! pip install cohere altair umap-learn -q
```
-```python PYTHON
+
+```python
+# Import the libraries
import cohere
import pandas as pd
import numpy as np
import altair as alt
+# Setup the Cohere client
co = cohere.Client("COHERE_API_KEY") # Get your API key: https://dashboard.cohere.com/api-keys
```
-The Cohere Generate endpoint generates text given an input, called “prompt”. The prompt provides a context of what we want the model to generate text. To illustrate this, let’s start with a simple prompt as the input.
+# 1 - Generating Text
+
+The Cohere Generate endpoint generates text given an input, called “prompt”. The prompt provides a context of what we want the model to generate text. To illustrate this, let’s start with a simple prompt as the input.
### Try a Simple Prompt
-```python PYTHON
+
+```python
+# Create a simple one-line prompt
prompt = "What is a Hello World program."
+# Generate text by calling the Chat endpoint
response = co.chat(
message=prompt,
model='command-r')
@@ -56,56 +70,62 @@ response = co.chat(
print(response.text)
```
-````
-A "Hello World" program is a traditional and simple program that is often used as an introduction to a new programming language. The program typically displays the message "Hello World" as its output. The concept of a "Hello World" program originated from the book *The C Programming Language* written by Kernighan and Ritchie, where the example program in the book displayed the message using the C programming language.
-
-The "Hello World" program serves as a basic and straightforward way to verify that your development environment is set up correctly and to familiarize yourself with the syntax and fundamentals of the programming language. It's a starting point for learning how to write and run programs in a new language.
-
-The program's simplicity makes it accessible to programmers of all skill levels, and it's often one of the first programs beginners write when learning to code. The exact implementation of a "Hello World" program varies depending on the programming language being used, but the core idea remains the same—to display the "Hello World" message.
-
-Here's how a "Hello World" program can be written in a few select languages:
-1. **C**:
-```c
-#include
-
-
-
+
+
+
+
+
+
We use the Embed endpoint to get the embeddings for each of these keywords.
-```python PYTHON
+
+```python
+# A function that classifies a list of inputs given the examples
def embed_text(texts, input_type):
"""
Turns a piece of text into embeddings
@@ -383,11 +444,13 @@ def embed_text(texts, input_type):
model="embed-english-v3.0",
input_type=input_type,
texts=texts)
-
+
return response.embeddings
```
-```python PYTHON
+
+```python
+# Get embeddings of all search terms
df["search_term_embeds"] = embed_text(texts=df["search_term"].tolist(),
input_type="search_document")
doc_embeds = np.array(df["search_term_embeds"].tolist())
@@ -397,16 +460,22 @@ doc_embeds = np.array(df["search_term_embeds"].tolist())
We’ll look at a couple of example applications. The first example is semantic search. Given a new query, our "search engine" must return the most similar FAQs, where the FAQs are the 50 search terms we uploaded earlier.
-```python PYTHON
+
+
+```python
+# Add a new query
query = "what is the history of hello world"
+# Get embeddings of the new query
query_embeds = embed_text(texts=[query],
input_type="search_query")[0]
```
We use cosine similarity to compare the similarity of the new query with each of the FAQs
-```python PYTHON
+
+```python
+# Calculate cosine similarity
from sklearn.metrics.pairwise import cosine_similarity
@@ -437,9 +506,12 @@ def get_similarity(target, candidates):
Finally, we display the top 5 FAQs that match the new query
-```python PYTHON
+
+```python
+# Get the similarity between the new query and existing queries
similarity = get_similarity(query_embeds,doc_embeds)
+# Display the top 5 FAQs
print("New query:")
print(new_query,'\n')
@@ -448,17 +520,16 @@ for idx,score in similarity[:5]:
print(f"Similarity: {score:.2f};", df.iloc[idx]["search_term"])
```
-```
-New query:
-what is the history of hello world
-
-Similar queries:
-Similarity: 0.58; how did hello world originate
-Similarity: 0.56; where did hello world come from
-Similarity: 0.54; why hello world
-Similarity: 0.53; why is hello world so famous
-Similarity: 0.53; what is hello world
-```
+ New query:
+ what is the history of hello world
+
+ Similar queries:
+ Similarity: 0.58; how did hello world originate
+ Similarity: 0.56; where did hello world come from
+ Similarity: 0.54; why hello world
+ Similarity: 0.53; why is hello world so famous
+ Similarity: 0.53; what is hello world
+
### Semantic Exploration
@@ -466,16 +537,21 @@ In the second example, we take the same idea as semantic search and take a broad
We'll use the same 50 top web search terms about Hello, World! There are different techniques we can use to compress the embeddings down to just 2 dimensions while retaining as much information as possible. We'll use a technique called UMAP. And once we can get it down to 2 dimensions, we can plot these embeddings on a 2D chart.
-```python PYTHON
+
+```python
+# Reduce the embeddings' dimensions to 2 using UMAP
import umap
-reducer = umap.UMAP(n_neighbors=49)
+reducer = umap.UMAP(n_neighbors=49)
umap_embeds = reducer.fit_transform(doc_embeds)
+# Add the 2 dimensions to the dataframe
df['x'] = umap_embeds[:,0]
df['y'] = umap_embeds[:,1]
```
-```python PYTHON
+
+```python
+# Plot the 2-dimension embeddings on a chart
chart = alt.Chart(df).mark_circle(size=500).encode(
x=
alt.X('x',
@@ -488,7 +564,7 @@ chart = alt.Chart(df).mark_circle(size=500).encode(
scale=alt.Scale(zero=False),
axis=alt.Axis(labels=False, ticks=False, domain=False)
),
-
+
tooltip=['search_term']
)
@@ -505,4 +581,27 @@ result = (chart + text).configure(background="#FDF7F0"
result
```
+
+
+
+
+
+
+
+
+
+
+```python
+
+```
diff --git a/fern/pages/cookbooks/long-form-general-strategies.mdx b/fern/pages/cookbooks/long-form-general-strategies.mdx
index e5dd17a9f..ca43311d6 100644
--- a/fern/pages/cookbooks/long-form-general-strategies.mdx
+++ b/fern/pages/cookbooks/long-form-general-strategies.mdx
@@ -1,9 +1,9 @@
---
-title: Long-Form Text Strategies with Cohere
+title: Long Form General Strategies
slug: /page/long-form-general-strategies
description: "This discusses ways of getting Cohere's LLM platform to perform well in generating long-form text."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, text comprehension, reading comprehension, AI, context windows"
---
@@ -14,12 +14,15 @@ import { CookbookHeader } from "../../components/cookbook-header";
authors={[
{
name: "Ania Bialas",
- imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/c5dc5a3-Ania.jpg",
- },
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/c5dc5a3-Ania.jpg"
+ }
]}
/>
-| - | search_term | -
|---|---|
| 0 | -how to print hello world in python | -
| 1 | -what is hello world | -
| 2 | -how do you write hello world in an alert box | -
| 3 | -how to print hello world in java | -
| 4 | -how to write hello world in eclipse | -
| + | search_term | +
|---|---|
| 0 | +how to print hello world in python | +
| 1 | +what is hello world | +
| 2 | +how do you write hello world in an alert box | +
| 3 | +how to print hello world in java | +
| 4 | +how to write hello world in eclipse | +
Large Language Models (LLMs) are becoming increasingly capable of comprehending text, among others excelling in document analysis. The new Cohere model, [Command-R](https://huggingface.co/CohereForAI/c4ai-command-r-v01), boasts a context length of 128k, which makes it particularly effective for such tasks. Nevertheless, even with the extended context window, some documents might be too lengthy to accommodate in full.
@@ -27,28 +30,230 @@ In this cookbook, we'll explore techniques to address cases when relevant inform
We'll show you three potential mitigation strategies: truncating the document, query-based retrieval, and a "text rank" approach we use internally at Cohere.
-## Summary
+### Table of content:
+1. [Getting started](#getting-started)
+2. [Approach 1: Truncate](#truncate)
+3. [Approach 2: Query Based Retrieval](#query-based-retrieval)
+4. [Approach 3: Text Rank](#text-rank)
+
+### Summary
+
+| Approach | Description | Pros | Cons | When to use? |
+|-----------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|
+| Truncation | Truncate the document to fit the context window. | - Simplicity of implementation(does not rely on extrenal infrastructure)| - Loses information at the end of the document | Utilize when all relevant information is contained
at the beginning of the document. | +| Query Based Retrieval| Utilize semantic similarity to retrieve text chunks
that are most relevant to the query. | - Focuses on sections directly relevant to
the query | - Relies on a semantic similarity algorithm.
- Might lose broader context | Employ when seeking specific
information within the text. | +| Text Rank | Apply graph theory to generate a cohesive set
of chunks that effectively represent the document. | - Preserves the broader picture. | - Might lose detailed information. | Utilize in summaries and when the question
requires broader context. | + + + + + + -| Approach | Description | Pros | Cons | When to use? | -| --------------------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | -| Truncation | Truncate the document to fit the context window. | - Simplicity of implementation
(does not rely on extrenal infrastructure) | - Loses information at the end of the document | Utilize when all relevant information is contained
at the beginning of the document. | -| Query Based Retrieval | Utilize semantic similarity to retrieve text chunks
that are most relevant to the query. | - Focuses on sections directly relevant to
the query | - Relies on a semantic similarity algorithm.
- Might lose broader context | Employ when seeking specific
information within the text. | -| Text Rank | Apply graph theory to generate a cohesive set
of chunks that effectively represent the document. | - Preserves the broader picture. | - Might lose detailed information. | Utilize in summaries and when the question
requires broader context. | - -## Getting Started [#getting-started] - -```python PYTHON -%%capture -!pip install cohere -!pip install python-dotenv -!pip install tokenizers -!pip install langchain -!pip install nltk -!pip install networkx -!pip install pypdf2 + + + + +```python +#################################################################################################### +# +# Uncomment if you need to install the following packages +# +#################################################################################################### +# %%capture +# !pip install cohere +# !pip install python-dotenv +# !pip install tokenizers +# !pip install langchain +# !pip install nltk +# !pip install networkx +# !pip install pypdf2 ``` -```python PYTHON + Collecting cohere + Downloading cohere-4.54-py3-none-any.whl (52 kB) + [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m52.8/52.8 kB[0m [31m1.7 MB/s[0m eta [36m0:00:00[0m + [?25hRequirement already satisfied: aiohttp<4.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (3.9.3) + Collecting backoff<3.0,>=2.0 (from cohere) + Downloading backoff-2.2.1-py3-none-any.whl (15 kB) + Collecting fastavro<2.0,>=1.8 (from cohere) + Downloading fastavro-1.9.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB) + [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m3.1/3.1 MB[0m [31m33.1 MB/s[0m eta [36m0:00:00[0m + [?25hCollecting importlib_metadata<7.0,>=6.0 (from cohere) + Downloading importlib_metadata-6.11.0-py3-none-any.whl (23 kB) + Requirement already satisfied: requests<3.0.0,>=2.25.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.31.0) + Requirement already satisfied: urllib3<3,>=1.26 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.0.7) + Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.3.1) + Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (23.2.0) + Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.4.1) + Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (6.0.5) + Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.9.4) + Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (4.0.3) + Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib_metadata<7.0,>=6.0->cohere) (3.17.0) + Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.3.2) + Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.6) + Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (2024.2.2) + [31mERROR: Operation cancelled by user[0m[31m + [0mTraceback (most recent call last): + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper + status = run_func(*args) + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py", line 242, in wrapper + return func(self, options, args) + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/commands/install.py", line 324, in run + session = self.get_default_session(options) + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py", line 98, in get_default_session + self._session = self.enter_context(self._build_session(options)) + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py", line 125, in _build_session + session = PipSession( + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py", line 342, in __init__ + self.headers["User-Agent"] = user_agent() + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py", line 175, in user_agent + setuptools_dist = get_default_environment().get_distribution("setuptools") + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py", line 75, in get_default_environment + return select_backend().Environment.default() + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py", line 63, in select_backend + from . import pkg_resources + File "/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/pkg_resources.py", line 8, in
+
+
+ True
+
+
+
+# Example 1 - Basic Multilingual Search
+
+
### Import a list of documents
-```python PYTHON
+
+```python
+# Import documents
import tensorflow_datasets as tfds
dataset = tfds.load('trec', split='train')
texts = [item['text'].decode('utf-8') for item in tfds.as_numpy(dataset)]
print(f"Number of documents: {len(texts)}")
```
-```txt title="Output"
-Downloading and preparing dataset 350.79 KiB (download: 350.79 KiB, generated: 636.90 KiB, total: 987.69 KiB) to /root/tensorflow_datasets/trec/1.0.0...
+ Downloading and preparing dataset 350.79 KiB (download: 350.79 KiB, generated: 636.90 KiB, total: 987.69 KiB) to /root/tensorflow_datasets/trec/1.0.0...
-Dl Completed...: 0 url [00:00, ? url/s]
+ Dl Completed...: 0 url [00:00, ? url/s]
-Dl Size...: 0 MiB [00:00, ? MiB/s]
+ Dl Size...: 0 MiB [00:00, ? MiB/s]
-Extraction completed...: 0 file [00:00, ? file/s]
+ Extraction completed...: 0 file [00:00, ? file/s]
-Generating splits...: 0%| | 0/2 [00:00
+
+# Example 2 - Search-Based Question Answering
+
+
## Add an article and chunk it into smaller passages
-```python PYTHON
-!wget 'https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F' -O steve-jobs-commencement.txt
-```
+```python
+# We'll use Steve Jobs' Stanford University commencement address as the example. Link: https://news.stanford.edu/2005/06/12/youve-got-find-love-jobs-says/
-```txt title="Output"
---2023-06-08 06:11:19-- https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F
-Resolving docs.google.com (docs.google.com)... 74.125.200.101, 74.125.200.138, 74.125.200.102, ...
-Connecting to docs.google.com (docs.google.com)|74.125.200.101|:443... connected.
-HTTP request sent, awaiting response... 303 See Other
-Location: https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc [following]
-Warning: wildcards not supported in HTTP.
---2023-06-08 06:11:19-- https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc
-Resolving doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)... 74.125.130.132, 2404:6800:4003:c01::84
-Connecting to doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)|74.125.130.132|:443... connected.
-HTTP request sent, awaiting response... 200 OK
-Length: 11993 (12K) [text/plain]
-Saving to: ‘steve-jobs-commencement.txt’
-
-steve-jobs-commence 100%[===================>] 11.71K --.-KB/s in 0s
-
-2023-06-08 06:11:20 (115 MB/s) - ‘steve-jobs-commencement.txt’ saved [11993/11993]
+!wget 'https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F' -O steve-jobs-commencement.txt
```
-```python PYTHON
+ --2023-06-08 06:11:19-- https://docs.google.com/uc?export=download&id=1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F
+ Resolving docs.google.com (docs.google.com)... 74.125.200.101, 74.125.200.138, 74.125.200.102, ...
+ Connecting to docs.google.com (docs.google.com)|74.125.200.101|:443... connected.
+ HTTP request sent, awaiting response... 303 See Other
+ Location: https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc [following]
+ Warning: wildcards not supported in HTTP.
+ --2023-06-08 06:11:19-- https://doc-0g-84-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/84t4moii9dmg08hmrh6rfpp8ecrjh6jq/1686204675000/12721472133292131824/*/1f1INWOfJrHTFmbyF_0be5b4u_moz3a4F?e=download&uuid=a26288c7-ad0c-4707-ae0b-72cb94c224dc
+ Resolving doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)... 74.125.130.132, 2404:6800:4003:c01::84
+ Connecting to doc-0g-84-docs.googleusercontent.com (doc-0g-84-docs.googleusercontent.com)|74.125.130.132|:443... connected.
+ HTTP request sent, awaiting response... 200 OK
+ Length: 11993 (12K) [text/plain]
+ Saving to: ‘steve-jobs-commencement.txt’
+
+ steve-jobs-commence 100%[===================>] 11.71K --.-KB/s in 0s
+
+ 2023-06-08 06:11:20 (115 MB/s) - ‘steve-jobs-commencement.txt’ saved [11993/11993]
+
+
+
+
+```python
loader = TextLoader("steve-jobs-commencement.txt")
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
@@ -223,14 +242,16 @@ texts = text_splitter.split_documents(documents)
## Embed the passages and store them in an index
-```python PYTHON
+
+```python
embeddings = CohereEmbeddings(model = "multilingual-22-12")
db = Qdrant.from_documents(texts, embeddings, location=":memory:", collection_name="my_documents", distance_func="Dot")
```
## Enter a question
-```python PYTHON
+
+```python
questions = [
"What did the author liken The Whole Earth Catalog to?",
"What was Reed College great at?",
@@ -242,7 +263,10 @@ questions = [
## Answer the question based on the most relevant documents
-```python PYTHON
+
+
+```python
+# Create our own prompt template
prompt_template = """Text: {context}
@@ -255,13 +279,14 @@ PROMPT = PromptTemplate(
)
```
-```python PYTHON
+
+```python
chain_type_kwargs = {"prompt": PROMPT}
-qa = RetrievalQA.from_chain_type(llm=Cohere(model="command", temperature=0),
- chain_type="stuff",
- retriever=db.as_retriever(),
- chain_type_kwargs=chain_type_kwargs,
+qa = RetrievalQA.from_chain_type(llm=Cohere(model="command", temperature=0),
+ chain_type="stuff",
+ retriever=db.as_retriever(),
+ chain_type_kwargs=chain_type_kwargs,
return_source_documents=True)
for question in questions:
@@ -279,100 +304,100 @@ for question in questions:
print(f"{idx+1}: {source_wrapped}")
```
-```txt title="Output"
-------------------------------------------------------------------------------------------------------------------------------------------------------
-
-Question: What did the author liken The Whole Earth Catalog to?
-Answer: It was sort of like Google in paperback form, 35 years before Google came along
-
-Sources:
-1: When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a
-fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960s, before
-personal computers and desktop publishing, so it was all made with typewriters, scissors and Polaroid cameras. It was sort of like Google in paperback
-form, 35 years before Google came along: It was
-2: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the
-mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find
-yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they
-signed off. Stay Hungry. Stay Foolish. And I have always
-3: idealistic, and overflowing with neat tools and great notions.
-4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
-------------------------------------------------------------------------------------------------------------------------------------------------------
-
-Question: What was Reed College great at?
-Answer: Reed College was great at calligraphy instruction.
-
-Sources:
-1: Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every
-drawer, was beautifully hand calligraphed. Because I had dropped out and didn’t have to take the normal classes, I decided to take a calligraphy class
-to learn how to do this. I learned about serif and sans serif typefaces, about varying the amount of space between different letter combinations,
-about what makes great typography great. It was
-2: I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why
-did I drop out?
-3: never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they
-do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backward 10 years
-later.
-4: OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the
-required classes that didn’t interest me, and begin dropping in on the ones that looked interesting.
-------------------------------------------------------------------------------------------------------------------------------------------------------
-
-Question: What was the author diagnosed with?
-Answer: The author was diagnosed with cancer.
-
-Sources:
-1: I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my
-intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed
-the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with
-surgery. I had the surgery and I’m fine now.
-2: About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn’t even know
-what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than
-three to six months. My doctor advised me to go home and get my affairs in order, which is doctor’s code for prepare to die. It means to try to tell
-your kids everything you thought you’d have the
-3: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the
-mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find
-yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they
-signed off. Stay Hungry. Stay Foolish. And I have always
-4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
-------------------------------------------------------------------------------------------------------------------------------------------------------
-
-Question: What is the key lesson from this article?
-Answer: The key lesson from this article is that you have to trust that the dots will somehow connect in your future. You have to trust in something -- your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.
-
-Sources:
-1: Again, you can’t connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect
-in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all
-the difference in my life. My second story is about love and loss.
-2: Remembering that I’ll be dead soon is the most important tool I’ve ever encountered to help me make the big choices in life. Because almost everything
-— all external expectations, all pride, all fear of embarrassment or failure — these things just fall away in the face of death, leaving only what is
-truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are
-already naked. There is no reason not to follow your
-3: Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s
-thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and
-intuition. They somehow already know what you truly want to become. Everything else is secondary.
-4: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton
-as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and
-I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple
-had not changed that one bit. I had been rejected,
-------------------------------------------------------------------------------------------------------------------------------------------------------
-
-Question: What did the article say about Michael Jackson?
-Answer: The text did not provide information about Michael Jackson.
-
-Sources:
-1: baby boy; do you want him?” They said: “Of course.” My biological mother later found out that my mother had never graduated from college and that my
-father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents
-promised that I would someday go to college.
-2: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
-3: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton
-as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and
-I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple
-had not changed that one bit. I had been rejected,
-4: This was the closest I’ve been to facing death, and I hope it’s the closest I get for a few more decades. Having lived through it, I can now say this
-to you with a bit more certainty than when death was a useful but purely intellectual concept:
-```
+ ------------------------------------------------------------------------------------------------------------------------------------------------------
+
+ Question: What did the author liken The Whole Earth Catalog to?
+ Answer: It was sort of like Google in paperback form, 35 years before Google came along
+
+ Sources:
+ 1: When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a
+ fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960s, before
+ personal computers and desktop publishing, so it was all made with typewriters, scissors and Polaroid cameras. It was sort of like Google in paperback
+ form, 35 years before Google came along: It was
+ 2: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the
+ mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find
+ yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they
+ signed off. Stay Hungry. Stay Foolish. And I have always
+ 3: idealistic, and overflowing with neat tools and great notions.
+ 4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
+ ------------------------------------------------------------------------------------------------------------------------------------------------------
+
+ Question: What was Reed College great at?
+ Answer: Reed College was great at calligraphy instruction.
+
+ Sources:
+ 1: Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every
+ drawer, was beautifully hand calligraphed. Because I had dropped out and didn’t have to take the normal classes, I decided to take a calligraphy class
+ to learn how to do this. I learned about serif and sans serif typefaces, about varying the amount of space between different letter combinations,
+ about what makes great typography great. It was
+ 2: I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why
+ did I drop out?
+ 3: never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they
+ do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backward 10 years
+ later.
+ 4: OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the
+ required classes that didn’t interest me, and begin dropping in on the ones that looked interesting.
+ ------------------------------------------------------------------------------------------------------------------------------------------------------
+
+ Question: What was the author diagnosed with?
+ Answer: The author was diagnosed with cancer.
+
+ Sources:
+ 1: I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my
+ intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed
+ the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with
+ surgery. I had the surgery and I’m fine now.
+ 2: About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn’t even know
+ what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than
+ three to six months. My doctor advised me to go home and get my affairs in order, which is doctor’s code for prepare to die. It means to try to tell
+ your kids everything you thought you’d have the
+ 3: Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the
+ mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find
+ yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they
+ signed off. Stay Hungry. Stay Foolish. And I have always
+ 4: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
+ ------------------------------------------------------------------------------------------------------------------------------------------------------
+
+ Question: What is the key lesson from this article?
+ Answer: The key lesson from this article is that you have to trust that the dots will somehow connect in your future. You have to trust in something -- your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.
+
+ Sources:
+ 1: Again, you can’t connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect
+ in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all
+ the difference in my life. My second story is about love and loss.
+ 2: Remembering that I’ll be dead soon is the most important tool I’ve ever encountered to help me make the big choices in life. Because almost everything
+ — all external expectations, all pride, all fear of embarrassment or failure — these things just fall away in the face of death, leaving only what is
+ truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are
+ already naked. There is no reason not to follow your
+ 3: Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s
+ thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and
+ intuition. They somehow already know what you truly want to become. Everything else is secondary.
+ 4: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton
+ as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and
+ I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple
+ had not changed that one bit. I had been rejected,
+ ------------------------------------------------------------------------------------------------------------------------------------------------------
+
+ Question: What did the article say about Michael Jackson?
+ Answer: The text did not provide information about Michael Jackson.
+
+ Sources:
+ 1: baby boy; do you want him?” They said: “Of course.” My biological mother later found out that my mother had never graduated from college and that my
+ father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents
+ promised that I would someday go to college.
+ 2: beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.
+ 3: I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down — that I had dropped the baton
+ as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and
+ I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple
+ had not changed that one bit. I had been rejected,
+ 4: This was the closest I’ve been to facing death, and I hope it’s the closest I get for a few more decades. Having lived through it, I can now say this
+ to you with a bit more certainty than when death was a useful but purely intellectual concept:
+
## Questions in French
-```python PYTHON
+
+```python
questions_fr = [
"À quoi se compare The Whole Earth Catalog ?",
"Dans quoi Reed College était-il excellent ?",
@@ -382,18 +407,22 @@ questions_fr = [
]
```
-```python PYTHON
+```python
+# import langchain
+# langchain.debug = False
```
-```python PYTHON
+
+```python
+# Generate the answer given the context
chain_type_kwargs = {"prompt": PROMPT}
-qa = RetrievalQA.from_chain_type(llm=Cohere(model="command", temperature=0),
- chain_type="stuff",
- retriever=db.as_retriever(),
- chain_type_kwargs=chain_type_kwargs,
+qa = RetrievalQA.from_chain_type(llm=Cohere(model="command", temperature=0),
+ chain_type="stuff",
+ retriever=db.as_retriever(),
+ chain_type_kwargs=chain_type_kwargs,
return_source_documents=True)
for question in questions_fr:
@@ -405,25 +434,29 @@ for question in questions_fr:
print(f"Answer: {result}")
```
-```txt title="Output"
---------------------
-
-Question: À quoi se compare The Whole Earth Catalog ?
-Answer: The Whole Earth Catalog was like Google in paperback form, 35 years before Google came along.
---------------------
-
-Question: Dans quoi Reed College était-il excellent ?
-Answer: Reed College offered the best calligraphy instruction in the country.
---------------------
-
-Question: De quoi l'auteur a-t-il été diagnostiqué ?
-Answer: The author was diagnosed with a very rare form of pancreatic cancer that is curable with surgery.
---------------------
-
-Question: Quelle est la leçon clé de cet article ?
-Answer: The key lesson of this article is that remembering that you will die soon is the most important tool to help one make the big choices in life.
---------------------
+ --------------------
+
+ Question: À quoi se compare The Whole Earth Catalog ?
+ Answer: The Whole Earth Catalog was like Google in paperback form, 35 years before Google came along.
+ --------------------
+
+ Question: Dans quoi Reed College était-il excellent ?
+ Answer: Reed College offered the best calligraphy instruction in the country.
+ --------------------
+
+ Question: De quoi l'auteur a-t-il été diagnostiqué ?
+ Answer: The author was diagnosed with a very rare form of pancreatic cancer that is curable with surgery.
+ --------------------
+
+ Question: Quelle est la leçon clé de cet article ?
+ Answer: The key lesson of this article is that remembering that you will die soon is the most important tool to help one make the big choices in life.
+ --------------------
+
+ Question: Que disait l'article sur Michael Jackson ?
+ Answer: The text does not contain the answer to the question.
+
+
+
+```python
-Question: Que disait l'article sur Michael Jackson ?
-Answer: The text does not contain the answer to the question.
```
diff --git a/fern/pages/cookbooks/pdf-extractor.mdx b/fern/pages/cookbooks/pdf-extractor.mdx
index bf24e7836..ed11bb545 100644
--- a/fern/pages/cookbooks/pdf-extractor.mdx
+++ b/fern/pages/cookbooks/pdf-extractor.mdx
@@ -3,7 +3,7 @@ title: PDF Extractor with Native Multi Step Tool Use
slug: /page/pdf-extractor
description: "This page describes how to create an AI agent able to extract information from PDFs."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, PDF extraction, LLMs, AI agents"
---
@@ -14,12 +14,14 @@ import { CookbookHeader } from "../../components/cookbook-header";
authors={[
{
name: "Jason Jung",
- imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/0803e3d-Jason_Jung.jpg",
- },
+ imageSrc: "https://fern-image-hosting.s3.amazonaws.com/cohere/0803e3d-Jason_Jung.jpg"
+ }
]}
/>
-
-We will focus on the evaluation of **Retrieve** and **Response** (or **Generation**), and present a set of metrics for each phase. We will deep dive into each metric, to give you a full understanding of how we evaluate models and why we do it this way, and provide code so you can repdroduce on your own data.
+In this notebook, we'll show you how to evaluate the output of a RAG system. The high-level RAG flow is depicted in the diagram below.
+
+
+We will focus on the evaluation of **Retrive** and **Response** (or **Generation**), and present a set of metrics for each phase. We will deep dive into each metric, to give you a full understanding of how we evaluate models and why we do it this way, and provide code so you can repdroduce on your own data.
To demonstrate the metrics, we will use data from the [Docugami's KG-RAG](https://github.com/docugami/KG-RAG-datasets/tree/main/sec-10-q/data/v1) dataset, a RAG dataset for financial 10Q filing reports. We will focus only on evaluation, without performing the actual Retrieval and response Generation steps.
@@ -39,22 +41,28 @@ To demonstrate the metrics, we will use data from the [Docugami's KG-RAG](https:
3. [Generation Evaluation](#generation-evaluation)
4. [Final Comments](#final-comments)
-## Getting Started [#getting-started]
+
+## Getting Started
Let's start by setting the environment and downloading the dataset.
-```python PYTHON
+
+```python
%%capture
!pip install llama-index cohere openai
!pip install mistralai
```
-```python PYTHON
+
+```python
# required imports
+import cohere
from getpass import getpass
import os
import re
+import json
import numpy as np
+import pandas as pd
from llama_index.core import SimpleDirectoryReader
from llama_index.core.llama_dataset import download_llama_dataset, LabelledRagDataset
from openai import Client
@@ -64,7 +72,8 @@ from mistralai.client import MistralClient
For Response evaluation, we will use an LLM as a judge.
Any LLM can be used for this goal, but because evaluation is a very challenging task, we recommend using powerful LLMs, possibly as an ensemble of models. In [previous work](https://arxiv.org/pdf/2303.16634.pdf), it has been shown that models tend to assign higher scores to their own output. Since we generated the answers in this notebook using `command-r`, we will not use it for evaluation. We will provide two alternatives, `gpt-4` and `mistral`. We set `gpt-4` as the default model because, as mentioned above, evaluation is challenging, and `gpt-4` is powerful enough to efficiently perform the task.
-```python PYTHON
+
+```python
# Get keys
openai_api_key = getpass("Enter your OpenAI API Key: ")
# uncomment if you want to use mistral
@@ -77,14 +86,19 @@ model = "gpt-4"
```
-```python PYTHON
+ Enter your OpenAI API Key: ··········
+
+
+
+```python
if model == "gpt-4":
client = Client(api_key=openai_api_key)
else:
client = MistralClient(api_key=mistral_api_key)
```
-```python PYTHON
+
+```python
# let's define a function to get the model's response for a given input
def get_response(model, client, prompt):
response = client.chat.completions.create(
@@ -94,7 +108,8 @@ def get_response(model, client, prompt):
return response.choices[0].message.content
```
-```python PYTHON
+
+```python
# load the DocugamiKgRagSec10Q dataset
if os.path.exists("./data/source_files") and os.path.exists("./data/rag_dataset.json"):
rag_dataset = LabelledRagDataset.from_json("./data/rag_dataset.json")
@@ -103,19 +118,24 @@ else:
rag_dataset, documents = download_llama_dataset("DocugamiKgRagSec10Q", "./data")
```
-## Retrieval Evaluation [#retrieval-evaluation]
+ Loading files: 100%|██████████| 20/20 [01:44<00:00, 5.21s/file]
+
+
+
+## Retrieval Evaluation
In the Retrieval phase, we evaluate the set of **retrieved documents** against the **golden documents** set.
We use three standard metrics to evaluate retrieval:
-- **Precision**: the proportion of returned documents that are relevant, according to the gold annotation
-- **Recall**: the proportion of relevant documents in the gold data found in the retrieved documents
-- **Mean Average Precision** (**MAP**): measures the capability of the retriever to return relevant documents at the top of the list
+* **Precision**: the proportion of returned documents that are relevant, according to the gold annotation
+* **Recall**: the proportion of relevant documents in the gold data found in the retrieved documents
+* **Mean Average Precision** (**MAP**): measures the capability of the retriever to return relevant documents at the top of the list
We implement these three metrics in the class below:
-```python PYTHON
+
+```python
class RetrievalEvaluator:
def compute_precision(self, retrieved_documents, golden_documents):
@@ -140,14 +160,16 @@ class RetrievalEvaluator:
results = {'precision': [precision],
'recall': [recall],
'map': [map]}
- for k,v in results.items():
- print(f"{k}: {v[0]}")
+ results = pd.DataFrame(results)
+ return results
+
```
Let's now see how to use the class above to compute the results on a single datapoint.
-```python PYTHON
+
+```python
# select the index of a single datapoint - the first one in the dataset
idx = 0
@@ -165,13 +187,13 @@ print(f'Golden docs: {golden_docs}')
print(f'Retrieved docs: {retrieved_docs}')
```
-```txt title="Output"
-Query: How has Apple's total net sales changed over time?
-Golden docs: ['2022 Q3 AAPL.pdf', '2023 Q1 AAPL.pdf', '2023 Q2 AAPL.pdf', '2023 Q3 AAPL.pdf']
-Retrieved docs: ['2022 Q3 AAPL.pdf', '2023 Q1 MSFT.pdf', '2023 Q1 AAPL.pdf']
-```
+ Query: How has Apple's total net sales changed over time?
+ Golden docs: ['2022 Q3 AAPL.pdf', '2023 Q1 AAPL.pdf', '2023 Q2 AAPL.pdf', '2023 Q3 AAPL.pdf']
+ Retrieved docs: ['2022 Q3 AAPL.pdf', '2023 Q1 MSFT.pdf', '2023 Q1 AAPL.pdf']
+
+
-```python PYTHON
+```python
# we can now instantiate the evaluator
evaluate_retrieval = RetrievalEvaluator()
@@ -180,32 +202,127 @@ evaluate_retrieval.run_evals(retrieved_docs,golden_docs)
```
-```txt title="Output"
-precision: 0.67
-recall: 0.5
-map: 0.83
-```
+
+
+
+
+
+
+
+
+
What are the figures above telling us?
-- Precision (0.67) tells us that 2 out of 3 of the retrieved docs are correct
-- Recall (0.5) means that 2 out of 4 relevant docs have been retrieved
-- MAP (0.83) is computed as the average of 1/1 (the highest ranked doc is correct) and 2/3 (the 2nd ranked doc is wrong, the 3rd is correct).
+* Precision (0.67) tells us that 2 out of 3 of the retrieved docs are correct
+* Recall (0.5) means that 2 out of 4 relevant docs have been retrieved
+* MAP (0.83) is computed as the average of 1/1 (the highest ranked doc is correct) and 2/3 (the 2nd ranked doc is wrong, the 3rd is correct).
While the example here focuses on a single datapoint, you can easily apply the same metrics to all your dataset and get the overall performance of your Retrieve phase.
-## Generation Evaluation [#generation-evaluation]
+
+## Generation Evaluation
Evaluating grounded generation (the second step of RAG) is notoriously difficult, because generations are usually complex and rich of information, and simply labelling an answer as "good" or "bad" is not enough.
-To overcome this issue, we first decompose complex answers into a set of basic _claims_, where a claim is any sentence or part of a sentence in the answer that expresses a verifiable fact. Subsequently, we check the validity of each claim independently, defining the overall quality of the answer based on the correctness of the claims it includes.
+To overcome this issue, we first decompose complex answers into a set of basic *claims*, where a claim is any sentence or part of a sentence in the answer that expresses a verifiable fact. Subsequently, we check the validity of each claim independently, defining the overall quality of the answer based on the correctness of the claims it includes.
We use claims to compute three metrics:
-- **Faithfulness**, which measures how many of the claims in the generated response are supported by the retrieved documents. This is a fundamental metric, as it tells us how _grounded_ in the documents the response is, and, contextually, it allows us to spot hallucinations
+* **Faithfulness**, which measures how many of the claims in the generated response are supported by the retrieved documents. This is a fundamental metric, as it tells us how *grounded* in the documents the response is, and, contextually, it allows us to spot hallucinations
-- **Correctness**, which checks which claims in the response also occur in the gold answer
+* **Correctness**, which checks which claims in the response also occur in the gold answer
-- And **Coverage**, by which we assess how many of the claims in the gold answer are included in the generated response.
+* And **Coverage**, by which we assess how many of the claims in the gold answer are included in the generated response.
Note that Faithfulness and Correctness share the exact same approach, the difference being that the former checks the claims against the supporting docs, while the latter against the golden answer.
Also, while Correctness is measuring the precision of the claims in the response, Coverage can be seen as complementary, as it measures recall.
@@ -214,7 +331,8 @@ Also, while Correctness is measuring the precision of the claims in the response
Let's now see how we implement the evaluation described above using LLMs. Let's start with **claim extraction**.
-```python PYTHON
+
+```python
# first, let's define a function which extracts the claims from a response
def extract_claims(query, response, model, client):
@@ -231,7 +349,8 @@ def extract_claims(query, response, model, client):
```
-```python PYTHON
+
+```python
# now, let's consider this answer, which we previously generated with command-r
response = "Apple's total net sales experienced a decline over the last year. The three-month period ended July 1, 2023, saw a total net sale of $81,797 million, which was a 1% decrease from the same period in 2022. The nine-month period ended July 1, 2023, fared slightly better, with a 3% decrease in net sales compared to the first nine months of 2022.\nThis downward trend continued into the three and six-month periods ending April 1, 2023. Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022."
@@ -242,22 +361,22 @@ claims = extract_claims(query, response, model, client)
print(f"List of claims extracted from the model's response:\n\n{claims}")
```
-```txt title="Output"
-List of claims extracted from the model's response:
+ List of claims extracted from the model's response:
+
+ - Apple's total net sales experienced a decline over the last year.
+ - The three-month period ended July 1, 2023, saw a total net sale of $81,797 million.
+ - This was a 1% decrease from the same period in 2022.
+ - The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022.
+ - The downward trend continued into the three and six-month periods ending April 1, 2023.
+ - Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.
-- Apple's total net sales experienced a decline over the last year.
-- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million.
-- This was a 1% decrease from the same period in 2022.
-- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022.
-- The downward trend continued into the three and six-month periods ending April 1, 2023.
-- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022.
-```
### Claim Assessment
Nice! now that we have the list of claims, we can go ahead and **assess the validity** of each claim.
-```python PYTHON
+
+```python
# Let's create a function that checks each claim against a reference text,
# which here we will call "context". As you will see, we will use different contexts,
# depending on the metric we want to compute.
@@ -282,7 +401,8 @@ def assess_claims(query, claims, context, model, client):
### Faithfulness
-```python PYTHON
+
+```python
# Let's start with Faithfulness: in this case, we want to assess the claims
# in the response against the retrieved documents (i.e., context = retrieved documents)
@@ -299,20 +419,20 @@ assessed_claims_faithfulness = assess_claims(query=query,
print(f"Assessment of the claims extracted from the model's response:\n\n{assessed_claims_faithfulness}")
```
-```txt title="Output"
-Assessment of the claims extracted from the model's response:
+ Assessment of the claims extracted from the model's response:
+
+ - Apple's total net sales experienced a decline over the last year. SUPPORTED=1
+ - The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
+ - This was a 1% decrease from the same period in 2022. SUPPORTED=1
+ - The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=1
+ - The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
+ - Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=1
-- Apple's total net sales experienced a decline over the last year. SUPPORTED=1
-- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
-- This was a 1% decrease from the same period in 2022. SUPPORTED=1
-- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=1
-- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
-- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=1
-```
Great, we now have an assessment for each of the claims: in the last step, we just need to use these assessments to define the final score.
-```python PYTHON
+
+```python
# given the list of claims and their label, compute the final score
# as the proportion of correct claims over the full list of claims
def get_final_score(claims_list):
@@ -322,20 +442,21 @@ def get_final_score(claims_list):
return round(score, 2)
```
-```python PYTHON
+
+```python
score_faithfulness = get_final_score(assessed_claims_faithfulness)
print(f'Faithfulness: {score_faithfulness}')
```
-```txt title="Output"
-Faithfulness: 1.0
-```
+ Faithfulness: 1.0
+
The final Faithfulness score is 1, which means that the model's response is fully grounded in the retrieved documents: that's a very good news :)
Before moving on, let's modify the model's response by adding a piece of information which is **not** grounded in any document, and re-compute Faithfulness.
-```python PYTHON
+
+```python
# let's mess up the century, changing 2022 to 1922
modified_response = response.replace('2022', '1922')
@@ -355,18 +476,17 @@ score_faithfulness_modified_claims = get_final_score(assessed_modified_claims)
print(f'Faithfulness: {score_faithfulness_modified_claims}')
```
-```txt title="Output"
-Assessment of the modified claims:
-
-- Apple's total net sales experienced a decline over the last year. SUPPORTED=1
-- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
-- This was a 1% decrease from the same period in 1922. SUPPORTED=0
-- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 1922. SUPPORTED=0
-- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
-- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 1922. SUPPORTED=0
+ Assessment of the modified claims:
+
+ - Apple's total net sales experienced a decline over the last year. SUPPORTED=1
+ - The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
+ - This was a 1% decrease from the same period in 1922. SUPPORTED=0
+ - The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 1922. SUPPORTED=0
+ - The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
+ - Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 1922. SUPPORTED=0
+
+ Faithfulness: 0.5
-Faithfulness: 0.5
-```
As you can see, by assessing claims one by one, we are able to spot **hallucinations**, that is, the (corrupted) cases in which the information provided by the model is not grounded in any of the retrieved documents.
@@ -374,7 +494,8 @@ As you can see, by assessing claims one by one, we are able to spot **hallucinat
As said, Faithfulness and Correctness share the same logic, the only difference being that we will check the claims against the gold answer. We can therefore repeat the process above, and just substitute the `context`.
-```python PYTHON
+
+```python
# let's get the gold answer from the dataset
golden_answer = rag_dataset[idx].reference_answer
@@ -391,56 +512,57 @@ assessed_claims_correctness = assess_claims(query=query,
print(f"Assess the claims extracted from the model's response against the golden answer:\n\n{assessed_claims_correctness}")
```
-```txt title="Output"
-Assess the claims extracted from the model's response against the golden answer:
+ Assess the claims extracted from the model's response against the golden answer:
+
+ - Apple's total net sales experienced a decline over the last year. SUPPORTED=1
+ - The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
+ - This was a 1% decrease from the same period in 2022. SUPPORTED=0
+ - The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=0
+ - The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
+ - Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=0
-- Apple's total net sales experienced a decline over the last year. SUPPORTED=1
-- The three-month period ended July 1, 2023, saw a total net sale of $81,797 million. SUPPORTED=1
-- This was a 1% decrease from the same period in 2022. SUPPORTED=0
-- The nine-month period ended July 1, 2023, had a 3% decrease in net sales compared to the first nine months of 2022. SUPPORTED=0
-- The downward trend continued into the three and six-month periods ending April 1, 2023. SUPPORTED=1
-- Apple's total net sales decreased by 3% and 4% respectively, compared to the same periods in 2022. SUPPORTED=0
-```
As mentioned above, automatic evaluation is a hard task, and even when using powerful models, claim assessment can present problems: for example, the third claim is labelled as 0, even if it might be inferred from the information in the gold answer.
-```python PYTHON
+
+```python
# we can now compute the final Correctness score
score_correctness = get_final_score(assessed_claims_correctness)
print(f'Correctness: {score_correctness}')
```
-```txt title="Output"
-Correctness: 0.5
-```
+ Correctness: 0.5
+
+
+For Correctness, we found that only half of the claims in the generated response are found in the gold answer. Note that this is not necessarily an issue: reference answers are often non-exhaustive, especially in dataset including open-ended questions, like the one we are considering in this post, and *both* the generated and golden answer can include relevant information.
-For Correctness, we found that only half of the claims in the generated response are found in the gold answer. Note that this is not necessarily an issue: reference answers are often non-exhaustive, especially in dataset including open-ended questions, like the one we are considering in this post, and _both_ the generated and golden answer can include relevant information.
### Coverage
-We finally move to Coverage. Remember that, in this case, we want to check how many of the claims _in the gold answer_ are included in the generated response. To do it, we first need to extract the claims from the gold answer.
+We finally move to Coverage. Remember that, in this case, we want to check how many of the claims *in the gold answer* are included in the generated response. To do it, we first need to extract the claims from the gold answer.
+
-```python PYTHON
+```python
# let's extract the golden claims
gold_claims = extract_claims(query, golden_answer, model, client)
print(f"List of claims extracted from the gold answer:\n\n{gold_claims}")
```
-```txt title="Output"
-List of claims extracted from the gold answer:
+ List of claims extracted from the gold answer:
+
+ - For the quarterly period ended June 25, 2022, the total net sales were $82,959 million.
+ - For the quarterly period ended December 31, 2022, the total net sales were $117,154 million.
+ - For the quarterly period ended April 1, 2023, the total net sales were $94,836 million.
+ - For the quarterly period ended July 1, 2023, the total net sales were $81,797 million.
+ - There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022.
+ - There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023.
-- For the quarterly period ended June 25, 2022, the total net sales were $82,959 million.
-- For the quarterly period ended December 31, 2022, the total net sales were $117,154 million.
-- For the quarterly period ended April 1, 2023, the total net sales were $94,836 million.
-- For the quarterly period ended July 1, 2023, the total net sales were $81,797 million.
-- There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022.
-- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023.
-```
Then, we check which of these claims is present in the response generated by the model.
-```python PYTHON
+
+```python
# note that in, this case, the context is the model's response
assessed_claims_coverage = assess_claims(query=query,
claims=gold_claims,
@@ -452,31 +574,31 @@ assessed_claims_coverage = assess_claims(query=query,
print(f"Assess which of the gold claims is in the model's response:\n\n{assessed_claims_coverage}")
```
-```txt title="Output"
-Assess which of the gold claims is in the model's response:
+ Assess which of the gold claims is in the model's response:
+
+ - For the quarterly period ended June 25, 2022, the total net sales were $82,959 million. SUPPORTED=0
+ - For the quarterly period ended December 31, 2022, the total net sales were $117,154 million. SUPPORTED=0
+ - For the quarterly period ended April 1, 2023, the total net sales were $94,836 million. SUPPORTED=0
+ - For the quarterly period ended July 1, 2023, the total net sales were $81,797 million. SUPPORTED=1
+ - There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022. SUPPORTED=0
+ - There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023. SUPPORTED=1
-- For the quarterly period ended June 25, 2022, the total net sales were $82,959 million. SUPPORTED=0
-- For the quarterly period ended December 31, 2022, the total net sales were $117,154 million. SUPPORTED=0
-- For the quarterly period ended April 1, 2023, the total net sales were $94,836 million. SUPPORTED=0
-- For the quarterly period ended July 1, 2023, the total net sales were $81,797 million. SUPPORTED=1
-- There was an increase in total net sales from the quarter ended June 25, 2022, to the quarter ended December 31, 2022. SUPPORTED=0
-- There was a decrease in total net sales in the quarters ended April 1, 2023, and July 1, 2023. SUPPORTED=1
-```
-```python PYTHON
+
+```python
# we compute the final Coverage score
score_coverage = get_final_score(assessed_claims_coverage)
print(f'Coverage: {score_coverage}')
```
-```txt title="Output"
-Coverage: 0.33
-```
+ Coverage: 0.33
+
The Coverage score is telling us that 1/3 of the information in the gold answer is present in the generated answer. This is a useful information, that, similarly to what said above regarding Correctness, can raise further questions, such as: is it acceptable to have diverging information in the generated answer? Is any crucial piece of information missing in the generated answer?
The answer to these questions is use case-specific, and has to be made by the end user: The claim-based approach implemented here supports the user by providing a clear and detailed view on what the model is assessing and how.
-## Final Comments [#final-comments]
+
+## Final Comments
RAG evaluation is a hard task, especially the evaluation of the generated response. In this notebook we offer a clear, robust and replicable approach to evaluation, on which you can build on to build your evaluation pipeline.
diff --git a/fern/pages/cookbooks/rag-with-chat-embed.mdx b/fern/pages/cookbooks/rag-with-chat-embed.mdx
index 20cedc9e5..6ccc53ca1 100644
--- a/fern/pages/cookbooks/rag-with-chat-embed.mdx
+++ b/fern/pages/cookbooks/rag-with-chat-embed.mdx
@@ -3,16 +3,23 @@ title: RAG With Chat Embed and Rerank via Pinecone
slug: /page/rag-with-chat-embed
description: "This page contains a basic tutorial on how to build a RAG-powered chatbot."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, retrieval-augmented generation, RAG, chatbot"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
+
+
+
+
+
+
+ | + | precision | +recall | +map | +
|---|---|---|---|
| 0 | +0.67 | +0.5 | +0.83 | +
+
+
+
+This notebook shows how to build a RAG-powered chatbot with Cohere's Chat endpoint. The chatbot can extract relevant information from external documents and produce verifiable, inline citations in its responses.
This application will use several Cohere API endpoints:
@@ -22,19 +29,14 @@ This application will use several Cohere API endpoints:
The diagram below provides an overview of what we’ll build.
-
+
Here is a summary of the steps involved.
Initial phase:
-
- **Step 0**: Ingest the documents – get documents, chunk, embed, and index.
For each user-chatbot interaction:
-
- **Step 1**: Get the user message
- **Step 2**: Call the Chat endpoint in query-generation mode
- If at least one query is generated
@@ -43,11 +45,15 @@ For each user-chatbot interaction:
- If no query is generated
- **Step 4**: Call the Chat endpoint in normal mode to generate a response
-```python PYTHON
+# Setup
+
+
+```python
! pip install cohere hnswlib unstructured python-dotenv -q
```
-```python PYTHON
+
+```python
import cohere
from pinecone import Pinecone, PodSpec
import uuid
@@ -60,7 +66,8 @@ co = cohere.Client("COHERE_API_KEY") # Get your API key here: https://dashboard.
pc = Pinecone(api_key="PINECONE_API_KEY") # (get API key at app.pinecone.io)
```
-```python PYTHON
+
+```python
import cohere
import os
import dotenv
@@ -73,9 +80,30 @@ pc = Pinecone(
```
-First, we define the list of documents we want to ingest and make available for retrieval. As an example, we'll use the contents from the first module of Cohere's _LLM University: What are Large Language Models?_.
-```python PYTHON
+```python
+# #@title Enable text wrapping in Google Colab
+# Uncomment the code below for Google Colab
+
+# from IPython.display import HTML, display
+
+# def set_css():
+# display(HTML('''
+#
+# '''))
+# get_ipython().events.register('pre_run_cell', set_css)
+```
+
+# Create a vector store for ingestion and retrieval
+
+First, we define the list of documents we want to ingest and make available for retrieval. As an example, we'll use the contents from the first module of Cohere's *LLM University: What are Large Language Models?*.
+
+
+```python
raw_documents = [
{
"title": "Text Embeddings",
@@ -92,24 +120,26 @@ raw_documents = [
]
```
-Usually the number of documents for practical applications is vast, and so we'll need to be able to search documents efficiently. This involves breaking the documents into chunks, generating embeddings, and indexing the embeddings, as shown in the image below.
+Usually the number of documents for practical applications is vast, and so we'll need to be able to search documents efficiently. This involves breaking the documents into chunks, generating embeddings, and indexing the embeddings, as shown in the image below.
-We implement this in the `Vectorstore` class below, which takes the `raw_documents` list as input. Three methods are immediately called when creating an object of the `Vectorstore` class:
+We implement this in the `Vectorstore` class below, which takes the `raw_documents` list as input. Three methods are immediately called when creating an object of the `Vectorstore` class:
-`load_and_chunk()`
-This method uses the `partition_html()` method from the `unstructured` library to load the documents from URL and break them into smaller chunks. Each chunk is turned into a dictionary object with three fields:
+`load_and_chunk()`
+This method uses the `partition_html()` method from the `unstructured` library to load the documents from URL and break them into smaller chunks. Each chunk is turned into a dictionary object with three fields:
- `title` - the web page’s title,
- `text` - the textual content of the chunk, and
-- `url` - the web page’s URL.
-
+- `url` - the web page’s URL.
+
+
`embed()`
-This method uses Cohere's `embed-english-v3.0` model to generate embeddings of the chunked documents. Since our documents will be used for retrieval, we set `input_type="search_document"`. We send the documents to the Embed endpoint in batches, because the endpoint has a limit of 96 documents per call.
+This method uses Cohere's `embed-english-v3.0` model to generate embeddings of the chunked documents. Since our documents will be used for retrieval, we set `input_type="search_document"`. We send the documents to the Embed endpoint in batches, because the endpoint has a limit of 96 documents per call.
`index()`
-This method uses the `hsnwlib` package to index the document chunk embeddings. This will ensure efficient similarity search during retrieval. Note that `hnswlib` uses a vector library, and we have chosen it for its simplicity.
+This method uses the `hsnwlib` package to index the document chunk embeddings. This will ensure efficient similarity search during retrieval. Note that `hnswlib` uses a vector library, and we have chosen it for its simplicity.
+
-```python PYTHON
+```python
class Vectorstore:
"""
A class representing a collection of documents indexed into a vectorstore.
@@ -194,10 +224,10 @@ class Vectorstore:
environment="gcp-starter"
)
)
-
+
# connect to index
self.idx = pc.Index(index_name)
-
+
batch_size = 128
ids = [str(i) for i in range(len(self.docs))]
@@ -241,9 +271,9 @@ class Vectorstore:
top_n=self.rerank_top_k,
model="rerank-english-v2.0",
)
-
+
docs_reranked = [res['matches'][result.index] for result in rerank_results.results]
-
+
for doc in docs_reranked:
docs_retrieved.append(doc['metadata'])
@@ -252,18 +282,19 @@ class Vectorstore:
In the code cell below, we initialize an instance of the `Vectorstore` class and pass in the `raw_documents` list as input.
-```python PYTHON
+
+```python
+# Create an instance of the Vectorstore class with the given sources
vectorstore = Vectorstore(raw_documents)
```
-```
-Loading documents...
-Embedding document chunks...
-Indexing documents...
-Indexing complete
-```
+ Loading documents...
+ Embedding document chunks...
+ Indexing documents...
+ Indexing complete
-The `Vectorstore` class also has a `retrieve()` method, which we'll use to retrieve relevant document chunks given a query (as in Step 3 in the diagram shared at the beginning of this notebook). This method has two components: (1) dense retrieval, and (2) reranking.
+
+The `Vectorstore` class also has a `retrieve()` method, which we'll use to retrieve relevant document chunks given a query (as in Step 3 in the diagram shared at the beginning of this notebook). This method has two components: (1) dense retrieval, and (2) reranking.
### Dense retrieval
@@ -273,9 +304,9 @@ Search is performed by the `knn_query()` method from the `hnswlib` library. Give
### Reranking
-After semantic search, we implement a reranking step. While our semantic search component is already highly capable of retrieving relevant sources, the [Rerank endpoint](https://cohere.com/rerank) provides an additional boost to the quality of the search results, especially for complex and domain-specific queries. It takes the search results and sorts them according to their relevance to the query.
+After semantic search, we implement a reranking step. While our semantic search component is already highly capable of retrieving relevant sources, the [Rerank endpoint](https://cohere.com/rerank) provides an additional boost to the quality of the search results, especially for complex and domain-specific queries. It takes the search results and sorts them according to their relevance to the query.
-We call the Rerank endpoint with the `co.rerank()` method and define the number of top reranked document chunks to retrieve using the attribute `self.rerank_top_k=3`. The model we use is `rerank-english-v2.0`.
+We call the Rerank endpoint with the `co.rerank()` method and define the number of top reranked document chunks to retrieve using the attribute `self.rerank_top_k=3`. The model we use is `rerank-english-v2.0`.
This method returns the top retrieved document chunks `chunks_retrieved` so that they can be passed to the chatbot.
@@ -283,38 +314,44 @@ In the code cell below, we check the document chunks that are retrieved for the
## Test Retrieval
-```python PYTHON
+
+```python
vectorstore.retrieve("multi-head attention definition")
```
-```
-[{'text': 'The attention step used in transformer models is actually much more powerful, and it’s called multi-head attention. In multi-head attention, several different embeddings are used to modify the vectors and add context to them. Multi-head attention has helped language models reach much higher levels of efficacy when processing and generating text.',
- 'title': 'Transformer Models',
- 'url': 'https://docs.cohere.com/docs/transformer-models'},
- {'text': "What you learned in this chapter is simple self-attention. However, we can do much better than that. There is a method called multi-head attention, in which one doesn't only consider one embedding, but several different ones. These are all obtained from the original by transforming it in different ways. Multi-head attention has been very successful at the task of adding context to text. If you'd like to learn more about the self and multi-head attention, you can check out the following two",
- 'title': 'The Attention Mechanism',
- 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'},
- {'text': 'Attention helps give context to each word, based on the other words in the sentence (or text).',
- 'title': 'Transformer Models',
- 'url': 'https://docs.cohere.com/docs/transformer-models'}]
-```
-Next, we implement a class to handle the interaction between the user and the chatbot. It takes an instance of the `Vectorstore` class as input.
-The `run()` method will be used to run the chatbot application. It begins with the logic for getting the user message, along with a way for the user to end the conversation.
-Based on the user message, the chatbot needs to decide if it needs to consult external information before responding. If so, the chatbot determines an optimal set of search queries to use for retrieval. When we call `co.chat()` with `search_queries_only=True`, the Chat endpoint handles this for us automatically.
+ [{'text': 'The attention step used in transformer models is actually much more powerful, and it’s called multi-head attention. In multi-head attention, several different embeddings are used to modify the vectors and add context to them. Multi-head attention has helped language models reach much higher levels of efficacy when processing and generating text.',
+ 'title': 'Transformer Models',
+ 'url': 'https://docs.cohere.com/docs/transformer-models'},
+ {'text': "What you learned in this chapter is simple self-attention. However, we can do much better than that. There is a method called multi-head attention, in which one doesn't only consider one embedding, but several different ones. These are all obtained from the original by transforming it in different ways. Multi-head attention has been very successful at the task of adding context to text. If you'd like to learn more about the self and multi-head attention, you can check out the following two",
+ 'title': 'The Attention Mechanism',
+ 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'},
+ {'text': 'Attention helps give context to each word, based on the other words in the sentence (or text).',
+ 'title': 'Transformer Models',
+ 'url': 'https://docs.cohere.com/docs/transformer-models'}]
+
+
+
+# Create a chatbot
-The generated queries can be accessed from the `search_queries` field of the object that is returned. Then, what happens next depends on how many queries are returned.
+Next, we implement a class to handle the interaction between the user and the chatbot. It takes an instance of the `Vectorstore` class as input.
-- If queries are returned, we call the `retrieve()` method of the Vectorstore object for the retrieval step. The retrieved document chunks are then passed to the Chat endpoint by adding a `documents` parameter when we call `co.chat()` again.
+The `run()` method will be used to run the chatbot application. It begins with the logic for getting the user message, along with a way for the user to end the conversation.
+
+Based on the user message, the chatbot needs to decide if it needs to consult external information before responding. If so, the chatbot determines an optimal set of search queries to use for retrieval. When we call `co.chat()` with `search_queries_only=True`, the Chat endpoint handles this for us automatically.
+
+The generated queries can be accessed from the `search_queries` field of the object that is returned. Then, what happens next depends on how many queries are returned.
+- If queries are returned, we call the `retrieve()` method of the Vectorstore object for the retrieval step. The retrieved document chunks are then passed to the Chat endpoint by adding a `documents` parameter when we call `co.chat()` again.
- Otherwise, if no queries are returned, we call the Chat endpoint another time, passing the user message and without needing to add any documents to the call.
In either case, we also pass the `conversation_id` parameter, which retains the interactions between the user and the chatbot in the same conversation thread. We also enable the `stream` parameter so we can stream the chatbot response.
-We then print the chatbot's response. In the case that the external information was used to generate a response, we also display citations.
+We then print the chatbot's response. In the case that the external information was used to generate a response, we also display citations.
-```python PYTHON
+
+```python
class Chatbot:
def __init__(self, vectorstore: Vectorstore):
"""
@@ -400,74 +437,123 @@ class Chatbot:
print(f"\n{'-'*100}\n")
```
-We can now run the chatbot. For this, we create the instance of `Chatbot` and run the chatbot by invoking the `run()` method.
-The format of each citation is:
+
+
+
+
+
+
+
+
+
+
+# Run the chatbot
+
+We can now run the chatbot. For this, we create the instance of `Chatbot` and run the chatbot by invoking the `run()` method.
+
+The format of each citation is:
- `start`: The starting point of a span where one or more documents are referenced
- `end`: The ending point of a span where one or more documents are referenced
- `text`: The text representing this span
- `document_ids`: The IDs of the documents being referenced (`doc_0` being the ID of the first document passed to the `documents` creating parameter in the endpoint call, and so on)
-```python PYTHON
+
+```python
+# Create an instance of the Chatbot class
chatbot = Chatbot(vectorstore)
+# Run the chatbot
chatbot.run()
```
-```
-Chatbot:
-Hello! What's your question? I'm here to help you in any way I can.
-----------------------------------------------------------------------------------------------------
-
-Retrieving information...
-Chatbot:
-Word embeddings associate words with lists of numbers, so that similar words are close to each other and dissimilar words are further away.
-Sentence embeddings do the same thing, but for sentences. Each sentence is associated with a vector of numbers in a coherent way, so that similar sentences are assigned similar vectors, and different sentences are given different vectors.
-
-CITATIONS:
-start=0 end=15 text='Word embeddings' document_ids=['doc_0']
-start=16 end=53 text='associate words with lists of numbers' document_ids=['doc_0']
-start=63 end=100 text='similar words are close to each other' document_ids=['doc_0']
-start=105 end=139 text='dissimilar words are further away.' document_ids=['doc_0']
-start=140 end=159 text='Sentence embeddings' document_ids=['doc_0', 'doc_2']
-start=160 end=177 text='do the same thing' document_ids=['doc_0', 'doc_2']
-start=198 end=211 text='Each sentence' document_ids=['doc_0', 'doc_2']
-start=215 end=250 text='associated with a vector of numbers' document_ids=['doc_0', 'doc_2']
-start=256 end=264 text='coherent' document_ids=['doc_2']
-start=278 end=295 text='similar sentences' document_ids=['doc_0', 'doc_2']
-start=300 end=324 text='assigned similar vectors' document_ids=['doc_0', 'doc_2']
-start=330 end=349 text='different sentences' document_ids=['doc_0', 'doc_2']
-start=354 end=378 text='given different vectors.' document_ids=['doc_0', 'doc_2']
-
-DOCUMENTS:
-{'id': 'doc_0', 'text': 'In the previous chapters, you learned about word and sentence embeddings and similarity between words and sentences. In short, a word embedding is a way to associate words with lists of numbers (vectors) in such a way that similar words are associated with numbers that are close by, and dissimilar words with numbers that are far away from each other. A sentence embedding does the same thing, but associating a vector to every sentence. Similarity is a way to measure how similar two words (or', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
-{'id': 'doc_1', 'text': 'Sentence embeddings\n\nSo word embeddings seem to be pretty useful, but in reality, human language is much more complicated than simply a bunch of words put together. Human language has structure, sentences, etc. How would one be able to represent, for instance, a sentence? Well, here’s an idea. How about the sums of scores of all the words? For example, say we have a word embedding that assigns the following scores to these words:\n\nNo: [1,0,0,0]\n\nI: [0,2,0,0]\n\nAm: [-1,0,1,0]\n\nGood: [0,0,1,3]', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}
-{'id': 'doc_2', 'text': 'This is where sentence embeddings come into play. A sentence embedding is just like a word embedding, except it associates every sentence with a vector full of numbers, in a coherent way. By coherent, I mean that it satisfies similar properties as a word embedding. For instance, similar sentences are assigned to similar vectors, different sentences are assigned to different vectors, and most importantly, each of the coordinates of the vector identifies some (whether clear or obscure) property of', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}
-
-----------------------------------------------------------------------------------------------------
-
-Retrieving information...
-Chatbot:
-The similarities between words and sentences are both quantitative measures of how close the two given items are. There are two types of similarities that can be defined: dot product similarity, and cosine similarity. These methods can determine how similar two words, or sentences, are.
-
-CITATIONS:
-start=54 end=75 text='quantitative measures' document_ids=['doc_0']
-start=79 end=88 text='how close' document_ids=['doc_0']
-start=124 end=133 text='two types' document_ids=['doc_0', 'doc_4']
-start=171 end=193 text='dot product similarity' document_ids=['doc_0', 'doc_4']
-start=199 end=217 text='cosine similarity.' document_ids=['doc_0', 'doc_4']
-start=236 end=257 text='determine how similar' document_ids=['doc_0', 'doc_4']
-
-DOCUMENTS:
-{'id': 'doc_0', 'text': 'Now that we know embeddings quite well, let’s move on to using them to find similarities. There are two types of similarities we’ll define in this post: dot product similarity and cosine similarity. Both are very similar and very useful to determine if two words (or sentences) are similar.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
-{'id': 'doc_1', 'text': 'But let me add some numbers to this reasoning to make it more clear. Imagine that we calculate similarities for the words in each sentence, and we get the following:\n\nThis similarity makes sense in the following ways:\n\nThe similarity between each word and itself is 1.\n\nThe similarity between any irrelevant word (“the”, “of”, etc.) and any other word is 0.\n\nThe similarity between “bank” and “river” is 0.11.\n\nThe similarity between “bank” and “money” is 0.25.', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
-{'id': 'doc_2', 'text': 'And the results are:\n\nThe similarity between sentences 1 and 2: 6738.2858668486715\n\nThe similarity between sentences 1 and 3: -122.22666955510499\n\nThe similarity between sentences 2 and 3: -3.494608113647928\n\nThese results certainly confirm our predictions. The similarity between sentences 1 and 2 is 6738, which is high. The similarities between sentences 1 and 3, and 2 and 3, are -122 and -3.5 (dot products are allowed to be negative too!), which are much lower.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
-{'id': 'doc_3', 'text': 'But let me add some numbers to this reasoning to make it more clear. Imagine that we calculate similarities for the words in each sentence, and we get the following:\n\nThis similarity makes sense in the following ways:\n\nThe similarity between each word and itself is 1.\n\nThe similarity between any irrelevant word (“the”, “of”, etc.) and any other word is 0.\n\nThe similarity between “bank” and “river” is 0.11.\n\nThe similarity between “bank” and “money” is 0.25.', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
-{'id': 'doc_4', 'text': 'Now that we know embeddings quite well, let’s move on to using them to find similarities. There are two types of similarities we’ll define in this post: dot product similarity and cosine similarity. Both are very similar and very useful to determine if two words (or sentences) are similar.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
-{'id': 'doc_5', 'text': 'And the results are:\n\nThe similarity between sentences 1 and 2: 6738.2858668486715\n\nThe similarity between sentences 1 and 3: -122.22666955510499\n\nThe similarity between sentences 2 and 3: -3.494608113647928\n\nThese results certainly confirm our predictions. The similarity between sentences 1 and 2 is 6738, which is high. The similarities between sentences 1 and 3, and 2 and 3, are -122 and -3.5 (dot products are allowed to be negative too!), which are much lower.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
-
-----------------------------------------------------------------------------------------------------
-
-Ending chat.
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Chatbot:
+ Hello! What's your question? I'm here to help you in any way I can.
+ ----------------------------------------------------------------------------------------------------
+
+ Retrieving information...
+ Chatbot:
+ Word embeddings associate words with lists of numbers, so that similar words are close to each other and dissimilar words are further away.
+ Sentence embeddings do the same thing, but for sentences. Each sentence is associated with a vector of numbers in a coherent way, so that similar sentences are assigned similar vectors, and different sentences are given different vectors.
+
+ CITATIONS:
+ start=0 end=15 text='Word embeddings' document_ids=['doc_0']
+ start=16 end=53 text='associate words with lists of numbers' document_ids=['doc_0']
+ start=63 end=100 text='similar words are close to each other' document_ids=['doc_0']
+ start=105 end=139 text='dissimilar words are further away.' document_ids=['doc_0']
+ start=140 end=159 text='Sentence embeddings' document_ids=['doc_0', 'doc_2']
+ start=160 end=177 text='do the same thing' document_ids=['doc_0', 'doc_2']
+ start=198 end=211 text='Each sentence' document_ids=['doc_0', 'doc_2']
+ start=215 end=250 text='associated with a vector of numbers' document_ids=['doc_0', 'doc_2']
+ start=256 end=264 text='coherent' document_ids=['doc_2']
+ start=278 end=295 text='similar sentences' document_ids=['doc_0', 'doc_2']
+ start=300 end=324 text='assigned similar vectors' document_ids=['doc_0', 'doc_2']
+ start=330 end=349 text='different sentences' document_ids=['doc_0', 'doc_2']
+ start=354 end=378 text='given different vectors.' document_ids=['doc_0', 'doc_2']
+
+ DOCUMENTS:
+ {'id': 'doc_0', 'text': 'In the previous chapters, you learned about word and sentence embeddings and similarity between words and sentences. In short, a word embedding is a way to associate words with lists of numbers (vectors) in such a way that similar words are associated with numbers that are close by, and dissimilar words with numbers that are far away from each other. A sentence embedding does the same thing, but associating a vector to every sentence. Similarity is a way to measure how similar two words (or', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
+ {'id': 'doc_1', 'text': 'Sentence embeddings\n\nSo word embeddings seem to be pretty useful, but in reality, human language is much more complicated than simply a bunch of words put together. Human language has structure, sentences, etc. How would one be able to represent, for instance, a sentence? Well, here’s an idea. How about the sums of scores of all the words? For example, say we have a word embedding that assigns the following scores to these words:\n\nNo: [1,0,0,0]\n\nI: [0,2,0,0]\n\nAm: [-1,0,1,0]\n\nGood: [0,0,1,3]', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}
+ {'id': 'doc_2', 'text': 'This is where sentence embeddings come into play. A sentence embedding is just like a word embedding, except it associates every sentence with a vector full of numbers, in a coherent way. By coherent, I mean that it satisfies similar properties as a word embedding. For instance, similar sentences are assigned to similar vectors, different sentences are assigned to different vectors, and most importantly, each of the coordinates of the vector identifies some (whether clear or obscure) property of', 'title': 'Text Embeddings', 'url': 'https://docs.cohere.com/docs/text-embeddings'}
+
+ ----------------------------------------------------------------------------------------------------
+
+ Retrieving information...
+ Chatbot:
+ The similarities between words and sentences are both quantitative measures of how close the two given items are. There are two types of similarities that can be defined: dot product similarity, and cosine similarity. These methods can determine how similar two words, or sentences, are.
+
+ CITATIONS:
+ start=54 end=75 text='quantitative measures' document_ids=['doc_0']
+ start=79 end=88 text='how close' document_ids=['doc_0']
+ start=124 end=133 text='two types' document_ids=['doc_0', 'doc_4']
+ start=171 end=193 text='dot product similarity' document_ids=['doc_0', 'doc_4']
+ start=199 end=217 text='cosine similarity.' document_ids=['doc_0', 'doc_4']
+ start=236 end=257 text='determine how similar' document_ids=['doc_0', 'doc_4']
+
+ DOCUMENTS:
+ {'id': 'doc_0', 'text': 'Now that we know embeddings quite well, let’s move on to using them to find similarities. There are two types of similarities we’ll define in this post: dot product similarity and cosine similarity. Both are very similar and very useful to determine if two words (or sentences) are similar.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
+ {'id': 'doc_1', 'text': 'But let me add some numbers to this reasoning to make it more clear. Imagine that we calculate similarities for the words in each sentence, and we get the following:\n\nThis similarity makes sense in the following ways:\n\nThe similarity between each word and itself is 1.\n\nThe similarity between any irrelevant word (“the”, “of”, etc.) and any other word is 0.\n\nThe similarity between “bank” and “river” is 0.11.\n\nThe similarity between “bank” and “money” is 0.25.', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
+ {'id': 'doc_2', 'text': 'And the results are:\n\nThe similarity between sentences 1 and 2: 6738.2858668486715\n\nThe similarity between sentences 1 and 3: -122.22666955510499\n\nThe similarity between sentences 2 and 3: -3.494608113647928\n\nThese results certainly confirm our predictions. The similarity between sentences 1 and 2 is 6738, which is high. The similarities between sentences 1 and 3, and 2 and 3, are -122 and -3.5 (dot products are allowed to be negative too!), which are much lower.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
+ {'id': 'doc_3', 'text': 'But let me add some numbers to this reasoning to make it more clear. Imagine that we calculate similarities for the words in each sentence, and we get the following:\n\nThis similarity makes sense in the following ways:\n\nThe similarity between each word and itself is 1.\n\nThe similarity between any irrelevant word (“the”, “of”, etc.) and any other word is 0.\n\nThe similarity between “bank” and “river” is 0.11.\n\nThe similarity between “bank” and “money” is 0.25.', 'title': 'The Attention Mechanism', 'url': 'https://docs.cohere.com/docs/the-attention-mechanism'}
+ {'id': 'doc_4', 'text': 'Now that we know embeddings quite well, let’s move on to using them to find similarities. There are two types of similarities we’ll define in this post: dot product similarity and cosine similarity. Both are very similar and very useful to determine if two words (or sentences) are similar.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
+ {'id': 'doc_5', 'text': 'And the results are:\n\nThe similarity between sentences 1 and 2: 6738.2858668486715\n\nThe similarity between sentences 1 and 3: -122.22666955510499\n\nThe similarity between sentences 2 and 3: -3.494608113647928\n\nThese results certainly confirm our predictions. The similarity between sentences 1 and 2 is 6738, which is high. The similarities between sentences 1 and 3, and 2 and 3, are -122 and -3.5 (dot products are allowed to be negative too!), which are much lower.', 'title': 'Similarity Between Words and Sentences', 'url': 'https://docs.cohere.com/docs/similarity-between-words-and-sentences'}
+
+ ----------------------------------------------------------------------------------------------------
+
+ Ending chat.
+
+
+
+```python
+
```
diff --git a/fern/pages/cookbooks/rerank-demo.mdx b/fern/pages/cookbooks/rerank-demo.mdx
index d656ba614..93c10fcd4 100644
--- a/fern/pages/cookbooks/rerank-demo.mdx
+++ b/fern/pages/cookbooks/rerank-demo.mdx
@@ -1,5 +1,5 @@
---
-title: Learn How Cohere's Rerank Models Work
+title: Demo of Rerank
slug: /page/rerank-demo
description: "This page contains a basic tutorial on how Cohere's ReRank models work and how to use them."
@@ -10,9 +10,11 @@ keywords: "Cohere, ReRank"
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
-Enterprise customers often store and handle information in relational databases but querying such databases effectively requires bespoke knowledge of the underlying database structure as well as strong SQL coding skills. One way to address these challenges is to build an LLM agent capable of generating and executing SQL queries based on natural language. For example, if a user asks: `what are the top 4 rows in table X`, the agent should be able to generate `SELECT * FROM X LIMIT 4`, execute this query and return the output to the user.
+
+## Motivation
+Enterprise customers often store and handle information in relational databases but querying such databases effectively requires bespoke knowledge of the underlying database structure as well as strong SQL coding skills. One way to address these challenges is to build an LLM agent capable of generating and executing SQL queries based on natural language. For example If a user asks: ``what are the top 4 rows in table X``, the agent should be able to generate ``SELECT * FROM X LIMIT 4``, execute this query and return the output to the user.
## Objective
+In this notebook we explore how to setup a sql agent using langchain and Cohere models. We explore two ways to achieve this, one using the [create_sql_agent](https://api.python.langchain.com/en/latest/sql_agent/langchain_cohere.sql_agent.agent.create_sql_agent.html) abstraction from langchain-cohere, and another using the [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over SQL Databases. We show how this can be done seamlessly with langchain's existing SQLDBToolkit.
-In this notebook we explore how to setup a [Cohere ReAct Agent](https://github.com/langchain-ai/langchain-cohere/blob/main/libs/cohere/langchain_cohere/cohere_agent.py) to answer questions over SQL Databases. We show how this can be done seamlessly with langchain's existing [SQLDBToolkit](https://python.langchain.com/v0.1/docs/integrations/toolkits/sql_database/).
+The [create_sql_agent](https://api.python.langchain.com/en/latest/sql_agent/langchain_cohere.sql_agent.agent.create_sql_agent.html) option is recommended for most use cases and can be setup without much hassle.
-# Toolkit Setup [#sec_step0]
+## Table of Contents
-```python PYTHON
-from langchain.agents import AgentExecutor
-from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
-from langchain_core.prompts import ChatPromptTemplate
-from langchain_cohere.chat_models import ChatCohere
+- [SQL Agent with create_sql_agent](#sec_step0)
+- [SQL Agent using Cohere ReAct Agent](#sec_step1)
+ - [Toolkit Setup](#sec_step2)
+ - [SQL Agent](#sec_step3)
+ - [SQL Agent with context](#sec_step4)
+
+
+# SQL Agent with create_sql_agent
+
+
+
+
+```python
+from langchain_cohere import ChatCohere, create_sql_agent
from langchain_community.utilities.sql_database import SQLDatabase
-from langchain_community.agent_toolkits import SQLDatabaseToolkit
import os
-import json
```
-```python PYTHON
+
+```python
+####################################################################################################
+#
# Uncomment if you need to install the following packages
+#
+####################################################################################################
+
#!pip install --quiet langchain langchain_cohere langchain_experimental --upgrade
```
-Langchain already has a SQLDBToolkit that consists of 4 tools to handle SQL query generation, execution and validation. To use this, you simply need to define your LLM and DB and pass these into the Toolkit.
-
-These are the following tools:
-
-- 'sql_db_query': executes SQL code on the database
-- 'sql_db_schema': returns the schema of tables given the list of tables
-- 'sql_db_list_tables': lists the tables in the database
-- 'sql_db_query_checker': validates the SQL query
-```python PYTHON
+```python
# load the cohere api key
os.environ["COHERE_API_KEY"] = ""
```
-```python PYTHON
+
+```python
DB_NAME='Chinook.db'
MODEL="command-r-plus"
+
llm = ChatCohere(model=MODEL, temperature=0.1,verbose=True)
db = SQLDatabase.from_uri(f"sqlite:///{DB_NAME}")
-toolkit = SQLDatabaseToolkit(db=db, llm=llm)
-context = toolkit.get_context()
-tools = toolkit.get_tools()
-
-print('**List of pre-defined Langchain Tools**')
-print([tool.name for tool in tools])
```
-```txt title="Output"
-**List of pre-defined Langchain Tools**
-['sql_db_query', 'sql_db_schema', 'sql_db_list_tables', 'sql_db_query_checker']
+
+```python
+agent_executor = create_sql_agent(llm, db=db, verbose=True)
```
-# SQL Agent [#sec_step1]
+The create_sql_agent abstraction comes with access to the SQLDBTookit, and presents itself as an easy-to-use low-code solution to create an agent to answer questions over a db knowledge source.
+
-We follow the general cohere react agent setup in Langchain to build our SQL agent.
-```python PYTHON
-# define the prompt template
-prompt = ChatPromptTemplate.from_template("{input}")
-# instantiate the ReAct agent
-agent = create_cohere_react_agent(
- llm=llm,
- tools=tools,
- prompt=prompt,
-)
-agent_executor = AgentExecutor(agent=agent,
- tools=tools,
- verbose=True,
- return_intermediate_steps=True
- )
-```
-```python PYTHON
-output=agent_executor.invoke({
+```python
+agent_executor.invoke({
"input": 'what tables are available?',
})
```
-```txt title="Output"
-[1m> Entering new AgentExecutor chain...[0m
-[32;1m[1;3m
-I will use the sql_db_list_tables tool to find out which tables are available.
-{'tool_name': 'sql_db_list_tables', 'parameters': {'tool_input': ''}}
-[0m[38;5;200m[1;3mAlbum, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track[0m[32;1m[1;3mRelevant Documents: 0
-Cited Documents: 0
-Answer: The following tables are available: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track.
-Grounded answer: The following tables are available:
+
In this cookbook, we will be demonstrating an approach we use for evaluating summarization tasks using LLM evaluation.
-# Get Started [#start]
+### Table of contents:
+1. [Get started](#start)
+2. [Construct the evaluation dataset](#dataset)
+3. [Build the evaluation framework](#eval-framework)
+5. [Run evaluations](#run-evals)
+
+
+
+# Get Started
You'll need a Cohere API key to run this notebook. If you don't have a key, head to https://cohere.com/ to generate your key.
-```python PYTHON
-!pip install cohere datasets --quiet
+
+```python
+# %%capture
+!pip install "cohere<5" datasets --quiet
```
-```python PYTHON
+
+```python
import json
import random
import re
@@ -33,6 +49,7 @@ from getpass import getpass
from datasets import load_dataset
import pandas as pd
+# Set up Cohere client
co_api_key = getpass("Enter your Cohere API key: ")
co_model = "command-r"
co = cohere.Client(api_key=co_api_key)
@@ -40,39 +57,29 @@ co = cohere.Client(api_key=co_api_key)
As test data, we'll use transcripts from the [QMSum dataset](https://github.com/Yale-LILY/QMSum). Note that in addition to the transcripts, this dataset also contains reference summaries -- we will use only the transcripts as our approach is reference-free.
-```python PYTHON
+
+```python
qmsum = load_dataset("MocktaiLEngineer/qmsum-processed", split="validation")
transcripts = [x for x in qmsum["meeting_transcript"] if x is not None]
```
-```txt title="Output"
-Generating train split: 0%| | 0/1095 [00:00
+
+# Construct the evaluation dataset
We are interested in evaluating summarization in real-world, enterprise use cases, which typically have two distinguishing features as compared to academic summarization benchmarks:
-
- Enterprise use cases often focus on specific summarization objectives, e.g. "summarize action items".
- Enterprise use cases often feature specific instruction constraints, e.g. "summarize in bullets with each bullet under 20 words".
Therefore, we must first create a dataset that contains diverse summarization prompts. We will do this programmatically by building prompts from their components, as defined below:
-
- Prompt = text (e.g. transcript to be summarized) + instruction
- Instruction = instruction objective (e.g. "summarize action items") + modifiers
- Modifiers = format/length modifiers (e.g. "use bullets") + style/tone modifiers (e.g. "do not mention names") + ...
First, we define the prompt that combines the text and instructions. Here, we use a very basic prompt:
-```python PYTHON
+
+```python
prompt_template = """## meeting transcript
{transcript}
@@ -82,13 +89,17 @@ prompt_template = """## meeting transcript
Next, we build the instructions. Because each instruction may have a different objective and modifiers, we track them using metadata. This will later be required for evaluation (i.e. to know what the prompt is asking).
-```python PYTHON
+
+```python
+# In this cookbook, we will only consider the following 2 objectives and 4 format/length modifiers
instruction_objectives = {
"general_summarization": "Summarize the meeting based on the transcript.",
"action_items": "What are the follow-up items based on the meeting transcript?",
}
+# Note that not all format/length modifiers are compatible with all objectives. For example, action
+# items are best suited for bullet format.
format_length_modifiers = {
"paragraphs_short": {
"text": "In paragraph form, output your response. Use at least 10 words and at most 50 words in total.",
@@ -133,7 +144,8 @@ format_length_modifiers = {
Let's combine the objectives and format/length modifiers to finish building the instructions.
-```python PYTHON
+
+```python
instructions = []
for obj_name, obj_text in instruction_objectives.items():
for mod_data in format_length_modifiers.values():
@@ -146,114 +158,125 @@ for obj_name, obj_text in instruction_objectives.items():
}
instructions.append(instruction)
+# View the first two instructions
print(json.dumps(instructions[:2], indent=4))
```
-```python title="Output"
-[
- {
- "instruction": "Summarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.",
- "eval_metadata": {
- "format": "paragraphs",
- "min_length": 10,
- "max_length": 50
+ [
+ {
+ "instruction": "Summarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.",
+ "eval_metadata": {
+ "format": "paragraphs",
+ "min_length": 10,
+ "max_length": 50
+ },
+ "objective": "general_summarization"
},
- "objective": "general_summarization"
- },
- {
- "instruction": "Summarize the meeting based on the transcript. Return the answer in the form of paragraphs. Make sure your answer is between 50 and 200 words long.",
- "eval_metadata": {
- "format": "paragraphs",
- "min_length": 50,
- "max_length": 200
- },
- "objective": "general_summarization"
- }
-]
-```
+ {
+ "instruction": "Summarize the meeting based on the transcript. Return the answer in the form of paragraphs. Make sure your answer is between 50 and 200 words long.",
+ "eval_metadata": {
+ "format": "paragraphs",
+ "min_length": 50,
+ "max_length": 200
+ },
+ "objective": "general_summarization"
+ }
+ ]
+
Finally, let's build the final prompts by semi-randomly pairing the instructions with transcripts from the QMSum dataset.
-```python PYTHON
+
+```python
data = pd.DataFrame(instructions)
+# Randomly shuffle the top 25% of transcripts by length, then assign them to the data
transcripts = sorted(transcripts, key=lambda x: len(x), reverse=True)[:int(len(transcripts) * 0.25)]
random.seed(42)
random.shuffle(transcripts)
data["transcript"] = transcripts[:len(data)]
+# Build the prompt
data["prompt"] = data.apply(lambda x: prompt_template.format(transcript=x["transcript"], instructions=x["instruction"]), axis=1)
```
-```python PYTHON
-data["transcript_token_len"] = [len(x) for x in co.batch_tokenize(data["transcript"].tolist(), model=co_model)]
-```
-```python PYTHON
-print(data["prompt"][0])
+```python
+# (Optional) Let's also check the token lengths of our prompts.
+data["transcript_token_len"] = [len(x) for x in co.batch_tokenize(data["transcript"].tolist(), model=co_model)]
```
-```txt title="Output"
-## meeting transcript
-PhD F: As opposed to the rest of us
-PhD D: Well comment OK I I remind that me my first objective eh in the project is to to study difference parameters to to find a a good solution to detect eh the overlapping zone in eh speech recorded But eh tsk comment ehhh comment In that way comment I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone And eh so eh I was eh I am transcribing the the first session and I I have found eh eh one thousand acoustic events eh besides the overlapping zones eh I I I mean the eh breaths eh aspiration eh eh talk eh eh clap eh comment I do not know what is the different names eh you use to to name the the pause n speech
-Grad G: Oh I do not think we ve been doing it at that level of detail So
-PhD D: Eh I I I do I do not need to to to mmm to m to label the the different acoustic but I prefer because eh I would like to to study if eh I I will find eh eh a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh eh acoustic events to nnn to eh to find what is the ehm the false eh the false eh hypothesis eh nnn which eh are produced when we use the the ehm this eh parameter eh I mean pitch eh eh difference eh feature
-PhD A: You know I think some of these that are the nonspeech overlapping events may be difficult even for humans to tell that there s two there I mean if it s a tapping sound you would not necessarily or you know something like that it would be it might be hard to know that it was two separate events
-Grad G: Well You were not talking about just overlaps were you ? You were just talking about acoustic events
-PhD D: I I I I t I t I talk eh about eh acoustic events in general but eh my my objective eh will be eh to study eh overlapping zone Eh ? comment n Eh in twelve minutes I found eh eh one thousand acoustic events
-Professor E: How many overlaps were there in it ? No no how many of them were the overlaps of speech though ?
-PhD D: How many ? Eh almost eh three hundred eh in one session in five eh in forty five minutes Alm Three hundred overlapping zone With the overlapping zone overlapping speech speech what eh different duration
-Postdoc B: Does this ? So if you had an overlap involving three people how many times was that counted ?
-PhD D: three people two people Eh I would like to consider eh one people with difference noise eh in the background be
-Professor E: No no but I think what she s asking is pause if at some particular for some particular stretch you had three people talking instead of two did you call that one event ?
-PhD D: Oh Oh I consider one event eh for th for that eh for all the zone This th I I I con I consider I consider eh an acoustic event the overlapping zone the period where three speaker or eh are talking together
-Grad G: So let s say me and Jane are talking at the same time and then Liz starts talking also over all of us How many events would that be ?
-PhD D: So I do not understand
-Grad G: So two people are talking comment and then a third person starts talking Is there an event right here ?
-PhD D: Eh no No no For me is the overlapping zone because because you you have s you have more one eh more one voice eh eh produced in a in in a moment
-Grad G: So i if two or more people are talking
-Professor E: OK So I think We just wanted to understand how you are defining it So then in the region between since there there is some continuous region in between regions where there is only one person speaking And one contiguous region like that you are calling an event Is it Are you calling the beginning or the end of it the event or are you calling the entire length of it the event ?
-PhD D: I consider the the nnn the nnn nnn eh the entirety eh eh all all the time there were the voice has overlapped This is the idea But eh I I do not distinguish between the the numbers of eh speaker I m not considering eh the the ehm eh the fact of eh eh for example what did you say ? Eh at first eh eh two talkers are eh speaking and eh eh a third person eh join to to that For me it s eh it s eh all overlap zone with eh several numbers of speakers is eh eh the same acoustic event Wi but without any mark between the zone of the overlapping zone with two speakers eh speaking together and the zone with the three speakers
-Postdoc B: That would j just be one
-PhD D: Eh with eh a beginning mark and the ending mark Because eh for me is the is the zone with eh some kind of eh distortion the spectral I do not mind By the moment by the moment
-Grad G: Well but But you could imagine that three people talking has a different spectral characteristic than two
-PhD D: I I do not but eh but eh I have to study comment What will happen in a general way
-Grad G: So You had to start somewhere
-PhD C: So there s a lot of overlap
-PhD D: I I do not know what eh will will happen with the
-Grad G: That s a lot of overlap
-Professor E: So again that s that s three three hundred in forty five minutes that are that are speakers just speakers
-Postdoc B: But a a a th
-Professor E: So that s about eight per minute
-Postdoc B: But a thousand events in twelve minutes that s
-PhD C: But that can include taps
-Postdoc B: Well but a thousand taps in eight minutes is a l in twelve minutes is a lot
-PhD D: I I con I consider I consider acoustic events eh the silent too
-Grad G: Silence starting or silence ending
-PhD D: silent ground to bec to detect eh because I consider acoustic event all the things are not eh speech In ge in in in a general point of view
-Professor E: OK so how many of those thousand were silence ?
-PhD F: Not speech not speech or too much speech
-Professor E: Right So how many of those thousand were silence silent sections ?
-PhD D: silent I I I I do not I I have not the eh I I would like to to do a stylistic study
-## instructions
-Summarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.
+```python
+# Let's examine one of the prompts
+print(data["prompt"][0])
```
-# Build the evaluation framework [#eval-framework]
+ ## meeting transcript
+ PhD F: As opposed to the rest of us
+ PhD D: Well comment OK I I remind that me my first objective eh in the project is to to study difference parameters to to find a a good solution to detect eh the overlapping zone in eh speech recorded But eh tsk comment ehhh comment In that way comment I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone And eh so eh I was eh I am transcribing the the first session and I I have found eh eh one thousand acoustic events eh besides the overlapping zones eh I I I mean the eh breaths eh aspiration eh eh talk eh eh clap eh comment I do not know what is the different names eh you use to to name the the pause n speech
+ Grad G: Oh I do not think we ve been doing it at that level of detail So
+ PhD D: Eh I I I do I do not need to to to mmm to m to label the the different acoustic but I prefer because eh I would like to to study if eh I I will find eh eh a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh eh acoustic events to nnn to eh to find what is the ehm the false eh the false eh hypothesis eh nnn which eh are produced when we use the the ehm this eh parameter eh I mean pitch eh eh difference eh feature
+ PhD A: You know I think some of these that are the nonspeech overlapping events may be difficult even for humans to tell that there s two there I mean if it s a tapping sound you would not necessarily or you know something like that it would be it might be hard to know that it was two separate events
+ Grad G: Well You were not talking about just overlaps were you ? You were just talking about acoustic events
+ PhD D: I I I I t I t I talk eh about eh acoustic events in general but eh my my objective eh will be eh to study eh overlapping zone Eh ? comment n Eh in twelve minutes I found eh eh one thousand acoustic events
+ Professor E: How many overlaps were there in it ? No no how many of them were the overlaps of speech though ?
+ PhD D: How many ? Eh almost eh three hundred eh in one session in five eh in forty five minutes Alm Three hundred overlapping zone With the overlapping zone overlapping speech speech what eh different duration
+ Postdoc B: Does this ? So if you had an overlap involving three people how many times was that counted ?
+ PhD D: three people two people Eh I would like to consider eh one people with difference noise eh in the background be
+ Professor E: No no but I think what she s asking is pause if at some particular for some particular stretch you had three people talking instead of two did you call that one event ?
+ PhD D: Oh Oh I consider one event eh for th for that eh for all the zone This th I I I con I consider I consider eh an acoustic event the overlapping zone the period where three speaker or eh are talking together
+ Grad G: So let s say me and Jane are talking at the same time and then Liz starts talking also over all of us How many events would that be ?
+ PhD D: So I do not understand
+ Grad G: So two people are talking comment and then a third person starts talking Is there an event right here ?
+ PhD D: Eh no No no For me is the overlapping zone because because you you have s you have more one eh more one voice eh eh produced in a in in a moment
+ Grad G: So i if two or more people are talking
+ Professor E: OK So I think We just wanted to understand how you are defining it So then in the region between since there there is some continuous region in between regions where there is only one person speaking And one contiguous region like that you are calling an event Is it Are you calling the beginning or the end of it the event or are you calling the entire length of it the event ?
+ PhD D: I consider the the nnn the nnn nnn eh the entirety eh eh all all the time there were the voice has overlapped This is the idea But eh I I do not distinguish between the the numbers of eh speaker I m not considering eh the the ehm eh the fact of eh eh for example what did you say ? Eh at first eh eh two talkers are eh speaking and eh eh a third person eh join to to that For me it s eh it s eh all overlap zone with eh several numbers of speakers is eh eh the same acoustic event Wi but without any mark between the zone of the overlapping zone with two speakers eh speaking together and the zone with the three speakers
+ Postdoc B: That would j just be one
+ PhD D: Eh with eh a beginning mark and the ending mark Because eh for me is the is the zone with eh some kind of eh distortion the spectral I do not mind By the moment by the moment
+ Grad G: Well but But you could imagine that three people talking has a different spectral characteristic than two
+ PhD D: I I do not but eh but eh I have to study comment What will happen in a general way
+ Grad G: So You had to start somewhere
+ PhD C: So there s a lot of overlap
+ PhD D: I I do not know what eh will will happen with the
+ Grad G: That s a lot of overlap
+ Professor E: So again that s that s three three hundred in forty five minutes that are that are speakers just speakers
+ Postdoc B: But a a a th
+ Professor E: So that s about eight per minute
+ Postdoc B: But a thousand events in twelve minutes that s
+ PhD C: But that can include taps
+ Postdoc B: Well but a thousand taps in eight minutes is a l in twelve minutes is a lot
+ PhD D: I I con I consider I consider acoustic events eh the silent too
+ Grad G: Silence starting or silence ending
+ PhD D: silent ground to bec to detect eh because I consider acoustic event all the things are not eh speech In ge in in in a general point of view
+ Professor E: OK so how many of those thousand were silence ?
+ PhD F: Not speech not speech or too much speech
+ Professor E: Right So how many of those thousand were silence silent sections ?
+ PhD D: silent I I I I do not I I have not the eh I I would like to to do a stylistic study
+
+ ## instructions
+ Summarize the meeting based on the transcript. In paragraph form, output your response. Use at least 10 words and at most 50 words in total.
+
+
+
+
+# Build the evaluation framework
We now setup the tools we will use for evaluation.
We use three criteria that are graded using LLMs:
-
- Completeness: checks if the summary includes all the important information from the original text that it should include
- Correctness: checks if there are any hallucinations or factual inaccuracies in the summary
- Conciseness: checks if the summary includes any unnecessary information or wordiness
In this cookbook, we will use Command-R to grade the completions. However, note that in practice, we typically use an ensemble of multiple LLM evaluators to reduce any bias.
-```python PYTHON
+
+```python
+# LLM evaluation uses an LLM to grade the completion. We pass the original prompt, completion, and a
+# criteria to the LLM, and it returns a score between 1 and 5. We will use the following template as
+# the grading prompt.
grading_prompt_template = """You are an AI grader that given a prompt, a completion, and a criterion, grades the completion based on the prompt and criterion. Below is a prompt, a completion, and a criterion with which to grade the completion. You need to respond according to the criterion instructions.
@@ -304,11 +327,12 @@ def score_llm(prompt: str, completion: str, criteria: str) -> int:
```
In addition, we have two criteria that are graded programmatically:
-
- Format: checks if the summary follows the format (e.g. bullets) that was requested in the prompt
- Length: checks if the summary follows the length that was requested in the prompt.
-```python PYTHON
+
+```python
+# Non-LLM checks
def score_format(completion: str, format_type: str) -> int:
"""
@@ -389,13 +413,16 @@ def _extract_markdown_bullets(text: str, include_bullet: bool = False) -> List[s
```
-# Run evaluations [#run-evals]
+
+
+# Run evaluations
Now that we have our evaluation dataset and defined our evaluation functions, let's run evaluations!
First, we generate completions to be graded. We will use Cohere's [Command-R](https://huggingface.co/CohereForAI/c4ai-command-r-v01) model, boasting a context length of 128K.
-```python PYTHON
+
+```python
completions = []
for prompt in data["prompt"]:
completion = co.chat(message=prompt, model="command-r", temperature=0.2).text
@@ -404,19 +431,25 @@ for prompt in data["prompt"]:
data["completion"] = completions
```
-```python PYTHON
+
+```python
+# Let's look at the completion for the example prompt we showed earlier
print(data["completion"][0])
```
- PhD D is transcribing recorded sessions to locate overlapping speech zones and categorizing them as acoustic events. The team discusses the parameters PhD D should use and how to define these events, considering the number of speakers and silence.
+ PhD student D presented their method for detecting and counting acoustic events in recorded speech sessions, including speech overlaps and silence. The team sought clarification on D's definitions and counted events, discussing the challenges of distinguishing between different types of events.
+
Let's grade the completions using our LLM and non-LLM checks.
-```python PYTHON
+
+```python
+# Score format
data["format_score"] = data.apply(
lambda x: score_format(x["completion"], x["eval_metadata"]["format"]), axis=1
)
+# Score length
data["length_score"] = data.apply(
lambda x: score_length(
x["completion"],
@@ -427,149 +460,337 @@ data["length_score"] = data.apply(
axis=1,
)
+# Score completeness
data["completeness_score"] = data.apply(
lambda x: score_llm(x["prompt"], x["completion"], criteria_completeness), axis=1
)
+# Score correctness
data["correctness_score"] = data.apply(
lambda x: score_llm(x["prompt"], x["completion"], criteria_correctness), axis=1
)
+# Score conciseness
data["conciseness_score"] = data.apply(
lambda x: score_llm(x["prompt"], x["completion"], criteria_conciseness), axis=1
)
```
-```python PYTHON
+
+```python
+# Let's examine the final evaluation results
data
```
-
-
-
-
+
+
diff --git a/fern/pages/cookbooks/text-classification-using-embeddings.mdx b/fern/pages/cookbooks/text-classification-using-embeddings.mdx
index 351b4f139..ae9f88ee8 100644
--- a/fern/pages/cookbooks/text-classification-using-embeddings.mdx
+++ b/fern/pages/cookbooks/text-classification-using-embeddings.mdx
@@ -3,22 +3,21 @@ title: Text Classification Using Embeddings
slug: /page/text-classification-using-embeddings
description: "This page discusses the creation of a text classification model using word vector embeddings."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, text classification, classification models, word vector embeddings"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
+'
+# Paste your API key here. Remember to not share it publicly
+api_key = '
+
+
+
+# Semantic Search
Data searching techniques focus on using keywords to retrieve text-based information. You can take this a level higher. Use search queries to determine the intent and contextual meaning. In this section, you’ll use Cohere to create embeddings for the search query. Use the embeddings to get the similarity with your dataset’s embeddings. The output is a list of similar AI papers.
First, create a function to get similarities between two embeddings. This will use the cosine similarity algorithm from the sci-kit learn library.
-```python PYTHON
+
+
+```python
from sklearn.metrics.pairwise import cosine_similarity
def get_similarity(target,candidates):
@@ -273,18 +312,24 @@ def get_similarity(target,candidates):
Next, create embeddings for the search query
-```python PYTHON
+
+```python
+# Add new query
new_query = "graph network strategies"
+# Get embeddings of the new query
new_query_embeds = get_embeddings(new_query)
```
Finally, check the similarity between the two embeddings. Display the top 10 similar papers using your result
-```python PYTHON
+
+```python
+# Get the similarity between the search query and existing queries
similarity = get_similarity(new_query_embeds,embeds[:sample])
+# View the top 5 articles
print('Query:')
print(new_query,'\n')
@@ -294,222 +339,225 @@ for idx,sim in similarity:
```
-```
-Query:
-graph network strategies
-
-Similar queries:
-Similarity: 0.49; amp chain graphs: minimal separators and structure learning algorithms
-Similarity: 0.46; pure nash equilibria in resource graph games
-Similarity: 0.44; general value function networks
-Similarity: 0.42; on the online coalition structure generation problem
-Similarity: 0.42; efficient local search based on dynamic connectivity maintenance for minimum connected dominating set
-Similarity: 0.42; graph kernels: a survey
-Similarity: 0.39; rwne: a scalable random-walk based network embedding framework with personalized higher-order proximity preserved
-Similarity: 0.39; the petlon algorithm to plan efficiently for task-level-optimal navigation
-Similarity: 0.38; election manipulation on social networks: seeding, edge removal, edge addition
-Similarity: 0.38; a semi-exact algorithm for quickly computing a maximum weight clique in large sparse graphs
-Similarity: 0.37; probabilistic temporal networks with ordinary distributions: theory, robustness and expected utility
-Similarity: 0.36; adaptive greedy versus non-adaptive greedy for influence maximization
-Similarity: 0.35; classifier chains: a review and perspectives
-Similarity: 0.35; learning bayesian networks under sparsity constraints: a parameterized complexity analysis
-Similarity: 0.34; optimally deceiving a learning leader in stackelberg games
-Similarity: 0.34; planning high-level paths in hostile, dynamic, and uncertain environments
-Similarity: 0.34; computational complexity of computing symmetries in finite-domain planning
-Similarity: 0.33; on the indecisiveness of kelly-strategyproof social choice functions
-Similarity: 0.33; qualitative numeric planning: reductions and complexity
-Similarity: 0.33; intelligence in strategic games
-Similarity: 0.33; steady-state planning in expected reward multichain mdps
-Similarity: 0.32; hybrid-order network consensus for distributed multi-agent systems
-Similarity: 0.32; evaluating strategic structures in multi-agent inverse reinforcement learning
-Similarity: 0.32; multi-agent advisor q-learning
-Similarity: 0.32; fairness in influence maximization through randomization
-Similarity: 0.32; a sufficient statistic for influence in structured multiagent environments
-Similarity: 0.32; constraint and satisfiability reasoning for graph coloring
-Similarity: 0.31; sum-of-products with default values: algorithms and complexity results
-Similarity: 0.31; contrastive explanations of plans through model restrictions
-Similarity: 0.31; zone path construction (zac) based approaches for effective real-time ridesharing
-Similarity: 0.31; an external knowledge enhanced graph-based neural network for sentence ordering
-Similarity: 0.31; finding and recognizing popular coalition structures
-Similarity: 0.30; constraint-based diversification of jop gadgets
-Similarity: 0.30; improved high dimensional discrete bayesian network inference using triplet region construction
-Similarity: 0.30; migrating techniques from search-based multi-agent path finding solvers to sat-based approach
-Similarity: 0.30; proactive dynamic distributed constraint optimization problems
-Similarity: 0.30; safe multi-agent pathfinding with time uncertainty
-Similarity: 0.30; a logic-based explanation generation framework for classical and hybrid planning problems
-Similarity: 0.30; scalable online planning for multi-agent mdps
-Similarity: 0.30; on sparse discretization for graphical games
-Similarity: 0.30; fond planning with explicit fairness assumptions
-Similarity: 0.29; jointly learning environments and control policies with projected stochastic gradient ascent
-Similarity: 0.29; computational benefits of intermediate rewards for goal-reaching policy learning
-Similarity: 0.29; two-phase multi-document event summarization on core event graphs
-Similarity: 0.29; efficient large-scale multi-drone delivery using transit networks
-Similarity: 0.29; fast adaptive non-monotone submodular maximization subject to a knapsack constraint
-Similarity: 0.29; path counting for grid-based navigation
-Similarity: 0.29; preferences single-peaked on a tree: multiwinner elections and structural results
-Similarity: 0.29; merge-and-shrink: a compositional theory of transformations of factored transition systems
-Similarity: 0.29; lilotane: a lifted sat-based approach to hierarchical planning
-Similarity: 0.29; cost-optimal planning, delete relaxation, approximability, and heuristics
-Similarity: 0.29; learning over no-preferred and preferred sequence of items for robust recommendation
-Similarity: 0.28; a few queries go a long way: information-distortion tradeoffs in matching
-Similarity: 0.28; constrained multiagent markov decision processes: a taxonomy of problems and algorithms
-Similarity: 0.28; contiguous cake cutting: hardness results and approximation algorithms
-Similarity: 0.28; online relaxation refinement for satisficing planning: on partial delete relaxation, complete hill-climbing, and novelty pruning
-Similarity: 0.28; playing codenames with language graphs and word embeddings
-Similarity: 0.28; a theoretical perspective on hyperdimensional computing
-Similarity: 0.28; reasoning with pcp-nets
-Similarity: 0.28; improving the effectiveness and efficiency of stochastic neighbour embedding with isolation kernel
-Similarity: 0.27; teaching people by justifying tree search decisions: an empirical study in curling
-Similarity: 0.27; learning optimal decision sets and lists with sat
-Similarity: 0.27; efficient multi-objective reinforcement learning via multiple-gradient descent with iteratively discovered weight-vector sets
-Similarity: 0.27; liquid democracy: an algorithmic perspective
-Similarity: 0.27; contrasting the spread of misinformation in online social networks
-Similarity: 0.27; on the computational complexity of non-dictatorial aggregation
-Similarity: 0.27; planning with critical section macros: theory and practice
-Similarity: 0.27; regarding goal bounding and jump point search
-Similarity: 0.27; using machine learning for decreasing state uncertainty in planning
-Similarity: 0.26; strategyproof mechanisms for additively separable and fractional hedonic games
-Similarity: 0.26; ranking sets of objects: the complexity of avoiding impossibility results
-Similarity: 0.26; socially responsible ai algorithms: issues, purposes, and challenges
-Similarity: 0.26; inductive logic programming at 30: a new introduction
-Similarity: 0.26; to regulate or not: a social dynamics analysis of an idealised ai race
-Similarity: 0.25; learning temporal causal sequence relationships from real-time time-series
-Similarity: 0.25; integrated offline and online decision making under uncertainty
-Similarity: 0.25; the computational complexity of understanding binary classifier decisions
-Similarity: 0.25; a survey of opponent modeling in adversarial domains
-Similarity: 0.25; cooperation and learning dynamics under wealth inequality and diversity in individual risk
-Similarity: 0.25; sunny-as2: enhancing sunny for algorithm selection
-Similarity: 0.25; game plan: what ai can do for football, and what football can do for ai
-Similarity: 0.25; approximating perfect recall when model checking strategic abilities: theory and applications
-Similarity: 0.25; efficient retrieval of matrix factorization-based top-k recommendations: a survey of recent approaches
-Similarity: 0.25; evolutionary dynamics and phi-regret minimization in games
-Similarity: 0.24; labeled bipolar argumentation frameworks
-Similarity: 0.24; optimal any-angle pathfinding on a sphere
-Similarity: 0.24; learning from disagreement: a survey
-Similarity: 0.24; on the cluster admission problem for cloud computing
-Similarity: 0.24; aggregation over metric spaces: proposing and voting in elections, budgeting, and legislation
-Similarity: 0.24; ordinal maximin share approximation for goods
-Similarity: 0.24; constraint solving approaches to the business-to-business meeting scheduling problem
-Similarity: 0.24; objective bayesian nets for integrating consistent datasets
-Similarity: 0.24; quantum mathematics in artificial intelligence
-Similarity: 0.24; task-aware verifiable rnn-based policies for partially observable markov decision processes
-Similarity: 0.23; samba: a generic framework for secure federated multi-armed bandits
-Similarity: 0.23; automated reinforcement learning (autorl): a survey and open problems
-Similarity: 0.23; generic constraint-based block modeling using constraint programming
-Similarity: 0.23; a comprehensive framework for learning declarative action models
-Similarity: 0.23; optimizing for interpretability in deep neural networks with tree regularization
-Similarity: 0.22; impact of imputation strategies on fairness in machine learning
-Similarity: 0.22; set-to-sequence methods in machine learning: a review
-Similarity: 0.22; the ai liability puzzle and a fund-based work-around
-Similarity: 0.22; agent-based markov modeling for improved covid-19 mitigation policies
-Similarity: 0.22; the parameterized complexity of motion planning for snake-like robots
-Similarity: 0.22; induction and exploitation of subgoal automata for reinforcement learning
-Similarity: 0.22; point at the triple: generation of text summaries from knowledge base triples
-Similarity: 0.22; trends in integration of vision and language research: a survey of tasks, datasets, and methods
-Similarity: 0.21; multi-document summarization with determinantal point process attention
-Similarity: 0.21; reward machines: exploiting reward function structure in reinforcement learning
-Similarity: 0.21; computing bayes-nash equilibria in combinatorial auctions with verification
-Similarity: 0.21; taking principles seriously: a hybrid approach to value alignment in artificial intelligence
-Similarity: 0.21; two-facility location games with minimum distance requirement
-Similarity: 0.21; structure from randomness in halfspace learning with the zero-one loss
-Similarity: 0.21; avoiding negative side effects of autonomous systems in the open world
-Similarity: 0.21; learning realistic patterns from visually unrealistic stimuli: generalization and data anonymization
-Similarity: 0.21; properties of switch-list representations of boolean functions
-Similarity: 0.21; conceptual modeling of explainable recommender systems: an ontological formalization to guide their design and development
-Similarity: 0.21; predicting decisions in language based persuasion games
-Similarity: 0.20; epidemioptim: a toolbox for the optimization of control policies in epidemiological models
-Similarity: 0.20; ffci: a framework for interpretable automatic evaluation of summarization
-Similarity: 0.20; analysis of the impact of randomization of search-control parameters in monte-carlo tree search
-Similarity: 0.20; output space entropy search framework for multi-objective bayesian optimization
-Similarity: 0.20; multiobjective tree-structured parzen estimator
-Similarity: 0.19; on the decomposition of abstract dialectical frameworks and the complexity of naive-based semantics
-Similarity: 0.19; hebo: pushing the limits of sample-efficient hyper-parameter optimisation
-Similarity: 0.19; superintelligence cannot be contained: lessons from computability theory
-Similarity: 0.19; goal recognition for deceptive human agents through planning and gaze
-Similarity: 0.19; dimensional inconsistency measures and postulates in spatio-temporal databases
-Similarity: 0.19; explainable deep learning: a field guide for the uninitiated
-Similarity: 0.19; multi-label classification neural networks with hard logical constraints
-Similarity: 0.19; declarative algorithms and complexity results for assumption-based argumentation
-Similarity: 0.19; worst-case bounds on power vs. proportion in weighted voting games with an application to false-name manipulation
-Similarity: 0.19; collie: continual learning of language grounding from language-image embeddings
-Similarity: 0.18; a word selection method for producing interpretable distributional semantic word vectors
-Similarity: 0.18; the bottleneck simulator: a model-based deep reinforcement learning approach
-Similarity: 0.18; metric-distortion bounds under limited information
-Similarity: 0.18; neural natural language generation: a survey on multilinguality, multimodality, controllability and learning
-Similarity: 0.18; on the evolvability of monotone conjunctions with an evolutionary mutation mechanism
-Similarity: 0.18; a metric space for point process excitations
-Similarity: 0.18; autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey
-Similarity: 0.18; the application of machine learning techniques for predicting match results in team sport: a review
-Similarity: 0.18; incremental event calculus for run-time reasoning
-Similarity: 0.18; a survey on the explainability of supervised machine learning
-Similarity: 0.18; madras : multi agent driving simulator
-Similarity: 0.17; maximin share allocations on cycles
-Similarity: 0.17; the rediscovery hypothesis: language models need to meet linguistics
-Similarity: 0.17; some inapproximability results of map inference and exponentiated determinantal point processes
-Similarity: 0.17; visually grounded models of spoken language: a survey of datasets, architectures and evaluation techniques
-Similarity: 0.17; the complexity landscape of outcome determination in judgment aggregation
-Similarity: 0.17; survey and evaluation of causal discovery methods for time series
-Similarity: 0.17; agent-based modeling for predicting pedestrian trajectories around an autonomous vehicle
-Similarity: 0.16; rethinking fairness: an interdisciplinary survey of critiques of hegemonic ml fairness approaches
-Similarity: 0.16; mapping the landscape of artificial intelligence applications against covid-19
-Similarity: 0.16; crossing the conversational chasm: a primer on natural language processing for multilingual task-oriented dialogue systems
-Similarity: 0.16; neural machine translation: a review
-Similarity: 0.16; viewpoint: ethical by designer - how to grow ethical designers of artificial intelligence
-Similarity: 0.16; core challenges in embodied vision-language planning
-Similarity: 0.16; neural character-level syntactic parsing for chinese
-Similarity: 0.16; marginal distance and hilbert-schmidt covariances-based independence tests for multivariate functional data
-Similarity: 0.16; instance-level update in dl-lite ontologies through first-order rewriting
-Similarity: 0.16; the societal implications of deep reinforcement learning
-Similarity: 0.16; benchmark and survey of automated machine learning frameworks
-Similarity: 0.16; experimental comparison and survey of twelve time series anomaly detection algorithms
-Similarity: 0.15; annotator rationales for labeling tasks in crowdsourcing
-Similarity: 0.15; representative committees of peers
-Similarity: 0.15; fine-grained prediction of political leaning on social media with unsupervised deep learning
-Similarity: 0.15; out of context: a new clue for context modeling of aspect-based sentiment analysis
-Similarity: 0.15; loss functions, axioms, and peer review
-Similarity: 0.15; on the tractability of shap explanations
-Similarity: 0.15; doubly robust crowdsourcing
-Similarity: 0.15; adversarial framework with certified robustness for time-series domain via statistical features
-Similarity: 0.14; on quantifying literals in boolean logic and its applications to explainable ai
-Similarity: 0.14; bribery and control in stable marriage
-Similarity: 0.14; finding the hardest formulas for resolution
-Similarity: 0.14; supervised visual attention for simultaneous multimodal machine translation
-Similarity: 0.14; a tight bound for stochastic submodular cover
-Similarity: 0.14; ethics and governance of artificial intelligence: evidence from a survey of machine learning researchers
-Similarity: 0.13; flexible bayesian nonlinear model configuration
-Similarity: 0.13; multilingual machine translation: deep analysis of language-specific encoder-decoders
-Similarity: 0.13; multilabel classification with partial abstention: bayes-optimal prediction under label independence
-Similarity: 0.13; viewpoint: ai as author bridging the gap between machine learning and literary theory
-Similarity: 0.13; confident learning: estimating uncertainty in dataset labels
-Similarity: 0.13; belief change and 3-valued logics: characterization of 19,683 belief change operators
-Similarity: 0.13; get out of the bag! silos in ai ethics education: unsupervised topic modeling analysis of global ai curricula
-Similarity: 0.12; on the distortion value of elections with abstention
-Similarity: 0.12; measuring the occupational impact of ai: tasks, cognitive abilities and ai benchmarks
-Similarity: 0.11; fair division of indivisible goods for a class of concave valuations
-Similarity: 0.10; admissibility in probabilistic argumentation
-Similarity: 0.10; recursion in abstract argumentation is hard --- on the complexity of semantics based on weak admissibility
-Similarity: 0.10; weighted first-order model counting in the two-variable fragment with counting quantifiers
-Similarity: 0.10; incompatibilities between iterated and relevance-sensitive belief revision
-Similarity: 0.10; automatic recognition of the general-purpose communicative functions defined by the iso 24617-2 standard for dialog act annotation
-Similarity: 0.09; nlp methods for extraction of symptoms from unstructured data for use in prognostic covid-19 analytic models
-Similarity: 0.09; welfare guarantees in schelling segregation
-Similarity: 0.09; casa: conversational aspect sentiment analysis for dialogue understanding
-Similarity: 0.09; a survey of algorithms for black-box safety validation of cyber-physical systems
-Similarity: 0.07; relevance in belief update
-Similarity: 0.06; on super strong eth
-Similarity: 0.06; image captioning as an assistive technology: lessons learned from vizwiz 2020 challenge
-Similarity: 0.03; confronting abusive language online: a survey from the ethical and human rights perspective
-```
+ Query:
+ graph network strategies
+
+ Similar queries:
+ Similarity: 0.49; amp chain graphs: minimal separators and structure learning algorithms
+ Similarity: 0.46; pure nash equilibria in resource graph games
+ Similarity: 0.44; general value function networks
+ Similarity: 0.42; on the online coalition structure generation problem
+ Similarity: 0.42; efficient local search based on dynamic connectivity maintenance for minimum connected dominating set
+ Similarity: 0.42; graph kernels: a survey
+ Similarity: 0.39; rwne: a scalable random-walk based network embedding framework with personalized higher-order proximity preserved
+ Similarity: 0.39; the petlon algorithm to plan efficiently for task-level-optimal navigation
+ Similarity: 0.38; election manipulation on social networks: seeding, edge removal, edge addition
+ Similarity: 0.38; a semi-exact algorithm for quickly computing a maximum weight clique in large sparse graphs
+ Similarity: 0.37; probabilistic temporal networks with ordinary distributions: theory, robustness and expected utility
+ Similarity: 0.36; adaptive greedy versus non-adaptive greedy for influence maximization
+ Similarity: 0.35; classifier chains: a review and perspectives
+ Similarity: 0.35; learning bayesian networks under sparsity constraints: a parameterized complexity analysis
+ Similarity: 0.34; optimally deceiving a learning leader in stackelberg games
+ Similarity: 0.34; planning high-level paths in hostile, dynamic, and uncertain environments
+ Similarity: 0.34; computational complexity of computing symmetries in finite-domain planning
+ Similarity: 0.33; on the indecisiveness of kelly-strategyproof social choice functions
+ Similarity: 0.33; qualitative numeric planning: reductions and complexity
+ Similarity: 0.33; intelligence in strategic games
+ Similarity: 0.33; steady-state planning in expected reward multichain mdps
+ Similarity: 0.32; hybrid-order network consensus for distributed multi-agent systems
+ Similarity: 0.32; evaluating strategic structures in multi-agent inverse reinforcement learning
+ Similarity: 0.32; multi-agent advisor q-learning
+ Similarity: 0.32; fairness in influence maximization through randomization
+ Similarity: 0.32; a sufficient statistic for influence in structured multiagent environments
+ Similarity: 0.32; constraint and satisfiability reasoning for graph coloring
+ Similarity: 0.31; sum-of-products with default values: algorithms and complexity results
+ Similarity: 0.31; contrastive explanations of plans through model restrictions
+ Similarity: 0.31; zone path construction (zac) based approaches for effective real-time ridesharing
+ Similarity: 0.31; an external knowledge enhanced graph-based neural network for sentence ordering
+ Similarity: 0.31; finding and recognizing popular coalition structures
+ Similarity: 0.30; constraint-based diversification of jop gadgets
+ Similarity: 0.30; improved high dimensional discrete bayesian network inference using triplet region construction
+ Similarity: 0.30; migrating techniques from search-based multi-agent path finding solvers to sat-based approach
+ Similarity: 0.30; proactive dynamic distributed constraint optimization problems
+ Similarity: 0.30; safe multi-agent pathfinding with time uncertainty
+ Similarity: 0.30; a logic-based explanation generation framework for classical and hybrid planning problems
+ Similarity: 0.30; scalable online planning for multi-agent mdps
+ Similarity: 0.30; on sparse discretization for graphical games
+ Similarity: 0.30; fond planning with explicit fairness assumptions
+ Similarity: 0.29; jointly learning environments and control policies with projected stochastic gradient ascent
+ Similarity: 0.29; computational benefits of intermediate rewards for goal-reaching policy learning
+ Similarity: 0.29; two-phase multi-document event summarization on core event graphs
+ Similarity: 0.29; efficient large-scale multi-drone delivery using transit networks
+ Similarity: 0.29; fast adaptive non-monotone submodular maximization subject to a knapsack constraint
+ Similarity: 0.29; path counting for grid-based navigation
+ Similarity: 0.29; preferences single-peaked on a tree: multiwinner elections and structural results
+ Similarity: 0.29; merge-and-shrink: a compositional theory of transformations of factored transition systems
+ Similarity: 0.29; lilotane: a lifted sat-based approach to hierarchical planning
+ Similarity: 0.29; cost-optimal planning, delete relaxation, approximability, and heuristics
+ Similarity: 0.29; learning over no-preferred and preferred sequence of items for robust recommendation
+ Similarity: 0.28; a few queries go a long way: information-distortion tradeoffs in matching
+ Similarity: 0.28; constrained multiagent markov decision processes: a taxonomy of problems and algorithms
+ Similarity: 0.28; contiguous cake cutting: hardness results and approximation algorithms
+ Similarity: 0.28; online relaxation refinement for satisficing planning: on partial delete relaxation, complete hill-climbing, and novelty pruning
+ Similarity: 0.28; playing codenames with language graphs and word embeddings
+ Similarity: 0.28; a theoretical perspective on hyperdimensional computing
+ Similarity: 0.28; reasoning with pcp-nets
+ Similarity: 0.28; improving the effectiveness and efficiency of stochastic neighbour embedding with isolation kernel
+ Similarity: 0.27; teaching people by justifying tree search decisions: an empirical study in curling
+ Similarity: 0.27; learning optimal decision sets and lists with sat
+ Similarity: 0.27; efficient multi-objective reinforcement learning via multiple-gradient descent with iteratively discovered weight-vector sets
+ Similarity: 0.27; liquid democracy: an algorithmic perspective
+ Similarity: 0.27; contrasting the spread of misinformation in online social networks
+ Similarity: 0.27; on the computational complexity of non-dictatorial aggregation
+ Similarity: 0.27; planning with critical section macros: theory and practice
+ Similarity: 0.27; regarding goal bounding and jump point search
+ Similarity: 0.27; using machine learning for decreasing state uncertainty in planning
+ Similarity: 0.26; strategyproof mechanisms for additively separable and fractional hedonic games
+ Similarity: 0.26; ranking sets of objects: the complexity of avoiding impossibility results
+ Similarity: 0.26; socially responsible ai algorithms: issues, purposes, and challenges
+ Similarity: 0.26; inductive logic programming at 30: a new introduction
+ Similarity: 0.26; to regulate or not: a social dynamics analysis of an idealised ai race
+ Similarity: 0.25; learning temporal causal sequence relationships from real-time time-series
+ Similarity: 0.25; integrated offline and online decision making under uncertainty
+ Similarity: 0.25; the computational complexity of understanding binary classifier decisions
+ Similarity: 0.25; a survey of opponent modeling in adversarial domains
+ Similarity: 0.25; cooperation and learning dynamics under wealth inequality and diversity in individual risk
+ Similarity: 0.25; sunny-as2: enhancing sunny for algorithm selection
+ Similarity: 0.25; game plan: what ai can do for football, and what football can do for ai
+ Similarity: 0.25; approximating perfect recall when model checking strategic abilities: theory and applications
+ Similarity: 0.25; efficient retrieval of matrix factorization-based top-k recommendations: a survey of recent approaches
+ Similarity: 0.25; evolutionary dynamics and phi-regret minimization in games
+ Similarity: 0.24; labeled bipolar argumentation frameworks
+ Similarity: 0.24; optimal any-angle pathfinding on a sphere
+ Similarity: 0.24; learning from disagreement: a survey
+ Similarity: 0.24; on the cluster admission problem for cloud computing
+ Similarity: 0.24; aggregation over metric spaces: proposing and voting in elections, budgeting, and legislation
+ Similarity: 0.24; ordinal maximin share approximation for goods
+ Similarity: 0.24; constraint solving approaches to the business-to-business meeting scheduling problem
+ Similarity: 0.24; objective bayesian nets for integrating consistent datasets
+ Similarity: 0.24; quantum mathematics in artificial intelligence
+ Similarity: 0.24; task-aware verifiable rnn-based policies for partially observable markov decision processes
+ Similarity: 0.23; samba: a generic framework for secure federated multi-armed bandits
+ Similarity: 0.23; automated reinforcement learning (autorl): a survey and open problems
+ Similarity: 0.23; generic constraint-based block modeling using constraint programming
+ Similarity: 0.23; a comprehensive framework for learning declarative action models
+ Similarity: 0.23; optimizing for interpretability in deep neural networks with tree regularization
+ Similarity: 0.22; impact of imputation strategies on fairness in machine learning
+ Similarity: 0.22; set-to-sequence methods in machine learning: a review
+ Similarity: 0.22; the ai liability puzzle and a fund-based work-around
+ Similarity: 0.22; agent-based markov modeling for improved covid-19 mitigation policies
+ Similarity: 0.22; the parameterized complexity of motion planning for snake-like robots
+ Similarity: 0.22; induction and exploitation of subgoal automata for reinforcement learning
+ Similarity: 0.22; point at the triple: generation of text summaries from knowledge base triples
+ Similarity: 0.22; trends in integration of vision and language research: a survey of tasks, datasets, and methods
+ Similarity: 0.21; multi-document summarization with determinantal point process attention
+ Similarity: 0.21; reward machines: exploiting reward function structure in reinforcement learning
+ Similarity: 0.21; computing bayes-nash equilibria in combinatorial auctions with verification
+ Similarity: 0.21; taking principles seriously: a hybrid approach to value alignment in artificial intelligence
+ Similarity: 0.21; two-facility location games with minimum distance requirement
+ Similarity: 0.21; structure from randomness in halfspace learning with the zero-one loss
+ Similarity: 0.21; avoiding negative side effects of autonomous systems in the open world
+ Similarity: 0.21; learning realistic patterns from visually unrealistic stimuli: generalization and data anonymization
+ Similarity: 0.21; properties of switch-list representations of boolean functions
+ Similarity: 0.21; conceptual modeling of explainable recommender systems: an ontological formalization to guide their design and development
+ Similarity: 0.21; predicting decisions in language based persuasion games
+ Similarity: 0.20; epidemioptim: a toolbox for the optimization of control policies in epidemiological models
+ Similarity: 0.20; ffci: a framework for interpretable automatic evaluation of summarization
+ Similarity: 0.20; analysis of the impact of randomization of search-control parameters in monte-carlo tree search
+ Similarity: 0.20; output space entropy search framework for multi-objective bayesian optimization
+ Similarity: 0.20; multiobjective tree-structured parzen estimator
+ Similarity: 0.19; on the decomposition of abstract dialectical frameworks and the complexity of naive-based semantics
+ Similarity: 0.19; hebo: pushing the limits of sample-efficient hyper-parameter optimisation
+ Similarity: 0.19; superintelligence cannot be contained: lessons from computability theory
+ Similarity: 0.19; goal recognition for deceptive human agents through planning and gaze
+ Similarity: 0.19; dimensional inconsistency measures and postulates in spatio-temporal databases
+ Similarity: 0.19; explainable deep learning: a field guide for the uninitiated
+ Similarity: 0.19; multi-label classification neural networks with hard logical constraints
+ Similarity: 0.19; declarative algorithms and complexity results for assumption-based argumentation
+ Similarity: 0.19; worst-case bounds on power vs. proportion in weighted voting games with an application to false-name manipulation
+ Similarity: 0.19; collie: continual learning of language grounding from language-image embeddings
+ Similarity: 0.18; a word selection method for producing interpretable distributional semantic word vectors
+ Similarity: 0.18; the bottleneck simulator: a model-based deep reinforcement learning approach
+ Similarity: 0.18; metric-distortion bounds under limited information
+ Similarity: 0.18; neural natural language generation: a survey on multilinguality, multimodality, controllability and learning
+ Similarity: 0.18; on the evolvability of monotone conjunctions with an evolutionary mutation mechanism
+ Similarity: 0.18; a metric space for point process excitations
+ Similarity: 0.18; autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey
+ Similarity: 0.18; the application of machine learning techniques for predicting match results in team sport: a review
+ Similarity: 0.18; incremental event calculus for run-time reasoning
+ Similarity: 0.18; a survey on the explainability of supervised machine learning
+ Similarity: 0.18; madras : multi agent driving simulator
+ Similarity: 0.17; maximin share allocations on cycles
+ Similarity: 0.17; the rediscovery hypothesis: language models need to meet linguistics
+ Similarity: 0.17; some inapproximability results of map inference and exponentiated determinantal point processes
+ Similarity: 0.17; visually grounded models of spoken language: a survey of datasets, architectures and evaluation techniques
+ Similarity: 0.17; the complexity landscape of outcome determination in judgment aggregation
+ Similarity: 0.17; survey and evaluation of causal discovery methods for time series
+ Similarity: 0.17; agent-based modeling for predicting pedestrian trajectories around an autonomous vehicle
+ Similarity: 0.16; rethinking fairness: an interdisciplinary survey of critiques of hegemonic ml fairness approaches
+ Similarity: 0.16; mapping the landscape of artificial intelligence applications against covid-19
+ Similarity: 0.16; crossing the conversational chasm: a primer on natural language processing for multilingual task-oriented dialogue systems
+ Similarity: 0.16; neural machine translation: a review
+ Similarity: 0.16; viewpoint: ethical by designer - how to grow ethical designers of artificial intelligence
+ Similarity: 0.16; core challenges in embodied vision-language planning
+ Similarity: 0.16; neural character-level syntactic parsing for chinese
+ Similarity: 0.16; marginal distance and hilbert-schmidt covariances-based independence tests for multivariate functional data
+ Similarity: 0.16; instance-level update in dl-lite ontologies through first-order rewriting
+ Similarity: 0.16; the societal implications of deep reinforcement learning
+ Similarity: 0.16; benchmark and survey of automated machine learning frameworks
+ Similarity: 0.16; experimental comparison and survey of twelve time series anomaly detection algorithms
+ Similarity: 0.15; annotator rationales for labeling tasks in crowdsourcing
+ Similarity: 0.15; representative committees of peers
+ Similarity: 0.15; fine-grained prediction of political leaning on social media with unsupervised deep learning
+ Similarity: 0.15; out of context: a new clue for context modeling of aspect-based sentiment analysis
+ Similarity: 0.15; loss functions, axioms, and peer review
+ Similarity: 0.15; on the tractability of shap explanations
+ Similarity: 0.15; doubly robust crowdsourcing
+ Similarity: 0.15; adversarial framework with certified robustness for time-series domain via statistical features
+ Similarity: 0.14; on quantifying literals in boolean logic and its applications to explainable ai
+ Similarity: 0.14; bribery and control in stable marriage
+ Similarity: 0.14; finding the hardest formulas for resolution
+ Similarity: 0.14; supervised visual attention for simultaneous multimodal machine translation
+ Similarity: 0.14; a tight bound for stochastic submodular cover
+ Similarity: 0.14; ethics and governance of artificial intelligence: evidence from a survey of machine learning researchers
+ Similarity: 0.13; flexible bayesian nonlinear model configuration
+ Similarity: 0.13; multilingual machine translation: deep analysis of language-specific encoder-decoders
+ Similarity: 0.13; multilabel classification with partial abstention: bayes-optimal prediction under label independence
+ Similarity: 0.13; viewpoint: ai as author bridging the gap between machine learning and literary theory
+ Similarity: 0.13; confident learning: estimating uncertainty in dataset labels
+ Similarity: 0.13; belief change and 3-valued logics: characterization of 19,683 belief change operators
+ Similarity: 0.13; get out of the bag! silos in ai ethics education: unsupervised topic modeling analysis of global ai curricula
+ Similarity: 0.12; on the distortion value of elections with abstention
+ Similarity: 0.12; measuring the occupational impact of ai: tasks, cognitive abilities and ai benchmarks
+ Similarity: 0.11; fair division of indivisible goods for a class of concave valuations
+ Similarity: 0.10; admissibility in probabilistic argumentation
+ Similarity: 0.10; recursion in abstract argumentation is hard --- on the complexity of semantics based on weak admissibility
+ Similarity: 0.10; weighted first-order model counting in the two-variable fragment with counting quantifiers
+ Similarity: 0.10; incompatibilities between iterated and relevance-sensitive belief revision
+ Similarity: 0.10; automatic recognition of the general-purpose communicative functions defined by the iso 24617-2 standard for dialog act annotation
+ Similarity: 0.09; nlp methods for extraction of symptoms from unstructured data for use in prognostic covid-19 analytic models
+ Similarity: 0.09; welfare guarantees in schelling segregation
+ Similarity: 0.09; casa: conversational aspect sentiment analysis for dialogue understanding
+ Similarity: 0.09; a survey of algorithms for black-box safety validation of cyber-physical systems
+ Similarity: 0.07; relevance in belief update
+ Similarity: 0.06; on super strong eth
+ Similarity: 0.06; image captioning as an assistive technology: lessons learned from vizwiz 2020 challenge
+ Similarity: 0.03; confronting abusive language online: a survey from the ethical and human rights perspective
+
Visualizing semantic search: https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/Visualizing_Text_Embeddings.ipynb
-Clustering is a process of grouping similar documents into clusters. It allows you to organize many documents into a smaller number of groups. As a result, you can discover emerging patterns in the documents. In this section, you will use the k-Means clustering algorithm to identify the top 5 clusters.
+# Text Clustering
+Clustering is a process of grouping similar documents into clusters. It allows you to organize many documents into a smaller number of groups. As a result, you can discover emerging patterns in the documents. In this section, you will use the k-Means clustering algorithm to identify the top 5 clusters.
First, import the k-means algorithm from the scikit-learn package. Then configure two variables: the number of clusters and a duplicate dataset.
-```python PYTHON
+
+
+```python
from sklearn.cluster import KMeans
+# Pick the number of clusters
df_clust = df_pc2.copy()
n_clusters=5
@@ -517,7 +565,9 @@ n_clusters=5
Next, initialize the k-means model and use it to fit the embeddings to create the clusters.
-```python PYTHON
+
+```python
+# Cluster the embeddings
kmeans_model = KMeans(n_clusters=n_clusters, random_state=0)
classes = kmeans_model.fit_predict(embeds).tolist()
print(classes)
@@ -525,17 +575,29 @@ df_clust['cluster'] = (list(map(str,classes)))
```
-```
-[2, 0, 3, 4, 4, 3, 1, 1, 0, 3, 0, 2, 0, 1, 1, 0, 0, 2, 0, 1, 3, 2, 1, 3, 0, 2, 2, 0, 2, 1, 1, 2, 2, 1, 0, 1, 1, 1, 2, 2, 2, 4, 3, 3, 3, 3, 2, 1, 2, 4, 3, 0, 2, 0, 1, 1, 0, 4, 0, 2, 2, 3, 1, 2, 4, 1, 2, 1, 4, 0, 3, 3, 4, 2, 0, 2, 2, 2, 0, 0, 0, 4, 1, 4, 1, 2, 0, 4, 1, 1, 4, 1, 4, 1, 4, 1, 0, 0, 4, 2, 4, 3, 4, 3, 2, 0, 2, 1, 1, 4, 2, 4, 2, 2, 0, 3, 1, 3, 2, 3, 1, 2, 0, 4, 4, 1, 0, 0, 4, 1, 1, 2, 2, 1, 2, 3, 0, 0, 1, 1, 1, 0, 4, 1, 4, 2, 4, 2, 4, 3, 2, 0, 1, 4, 1, 1, 2, 2, 0, 1, 1, 1, 1, 1, 1, 2, 2, 0, 2, 0, 4, 2, 4, 2, 1, 0, 3, 0, 1, 0, 2, 2, 1, 4, 1, 3, 4, 1, 0, 2, 1, 2, 0, 2, 4, 1, 4, 2, 2, 1, 0, 0, 1, 0, 2, 1, 0, 4, 1, 4, 0, 2, 1, 4, 1, 3, 2, 4, 2, 0, 1, 0, 3, 0, 2, 4, 1, 1, 3, 2, 3, 1, 3, 4, 2, 2, 0, 1, 1, 1, 4, 1, 0, 4, 3, 2, 2, 2, 2, 0, 1, 3, 1, 3, 4, 2, 4, 2, 1, 3]
-```
+ [2, 0, 3, 4, 4, 3, 1, 1, 0, 3, 0, 2, 0, 1, 1, 0, 0, 2, 0, 1, 3, 2, 1, 3, 0, 2, 2, 0, 2, 1, 1, 2, 2, 1, 0, 1, 1, 1, 2, 2, 2, 4, 3, 3, 3, 3, 2, 1, 2, 4, 3, 0, 2, 0, 1, 1, 0, 4, 0, 2, 2, 3, 1, 2, 4, 1, 2, 1, 4, 0, 3, 3, 4, 2, 0, 2, 2, 2, 0, 0, 0, 4, 1, 4, 1, 2, 0, 4, 1, 1, 4, 1, 4, 1, 4, 1, 0, 0, 4, 2, 4, 3, 4, 3, 2, 0, 2, 1, 1, 4, 2, 4, 2, 2, 0, 3, 1, 3, 2, 3, 1, 2, 0, 4, 4, 1, 0, 0, 4, 1, 1, 2, 2, 1, 2, 3, 0, 0, 1, 1, 1, 0, 4, 1, 4, 2, 4, 2, 4, 3, 2, 0, 1, 4, 1, 1, 2, 2, 0, 1, 1, 1, 1, 1, 1, 2, 2, 0, 2, 0, 4, 2, 4, 2, 1, 0, 3, 0, 1, 0, 2, 2, 1, 4, 1, 3, 4, 1, 0, 2, 1, 2, 0, 2, 4, 1, 4, 2, 2, 1, 0, 0, 1, 0, 2, 1, 0, 4, 1, 4, 0, 2, 1, 4, 1, 3, 2, 4, 2, 0, 1, 0, 3, 0, 2, 4, 1, 1, 3, 2, 3, 1, 3, 4, 2, 2, 0, 1, 1, 1, 4, 1, 0, 4, 3, 2, 2, 2, 2, 0, 1, 3, 1, 3, 4, 2, 4, 2, 1, 3]
+
Finally, plot a scatter plot to visualize the 5 clusters in our sample size.
-```python PYTHON
+
+
+
+```python
+# Plot on a chart
df_clust.columns = df_clust.columns.astype(str)
generate_chart(df_clust.iloc[:sample],'0','1',lbl='off',color='cluster',title='Clustering with 5 Clusters')
```
+
+
+
+
+
+
+
+# Conclusion
Let's recap the NLP tasks implemented in this tutorial. You’ve created word embeddings, perform a semantic search, and text clustering. Cohere’s platform provides NLP tools that are easy and intuitive to integrate. You can create digital experiences that support powerful NLP capabilities like text clustering. It’s easy to Register a Cohere account and gain access to an API key. New cohere accounts have $75 free credits for the first 3 months. It also offers a Pay-as-you-go Pricing Model that bills you upon usage.
+
diff --git a/fern/pages/cookbooks/wikipedia-search-with-weaviate.mdx b/fern/pages/cookbooks/wikipedia-search-with-weaviate.mdx
index 2cc444e28..5ecd80d9b 100644
--- a/fern/pages/cookbooks/wikipedia-search-with-weaviate.mdx
+++ b/fern/pages/cookbooks/wikipedia-search-with-weaviate.mdx
@@ -3,23 +3,34 @@ title: Wikipedia Semantic Search with Cohere + Weaviate
slug: /page/wikipedia-search-with-weaviate
description: "This page contains a description of building a Wikipedia-focused search engine with Cohere's LLM platform and the Weaviate vector database."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, Weaviate, vector databases"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-\""
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "6l1pDbAmsptW"
+ },
"source": [
"## Define tools\n",
"\n",
@@ -100,23 +89,25 @@
"- and/or directly answering [ this tool comes out of the box! ]\n",
"\n",
"Plus the model can self-reflect."
- ],
- "metadata": {
- "id": "6l1pDbAmsptW"
- }
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "_9riBTvIVnaN"
+ },
"source": [
"#### Web search\n",
"You can easily equip your agent with web search!"
- ],
- "metadata": {
- "id": "_9riBTvIVnaN"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "id": "6wMNTOGNVvwA"
+ },
+ "outputs": [],
"source": [
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
"\n",
@@ -131,50 +122,50 @@
"class TavilySearchInput(BaseModel):\n",
" query: str = Field(description=\"Query to search the internet with\")\n",
"internet_search.args_schema = TavilySearchInput"
- ],
- "metadata": {
- "id": "6wMNTOGNVvwA"
- },
- "execution_count": 3,
- "outputs": []
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "pax1SemzX6Kg"
+ },
"source": [
"#### Vector store\n",
"You can easily equip your agent with a vector store!"
- ],
- "metadata": {
- "id": "pax1SemzX6Kg"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "!pip --quiet install faiss-cpu tiktoken"
- ],
+ "execution_count": 4,
"metadata": {
- "id": "cc4d4pRFndYu",
"colab": {
"base_uri": "https://localhost:8080/"
},
+ "id": "cc4d4pRFndYu",
"outputId": "6478fd6b-5618-4d0b-8928-44f78e3ca7e6"
},
- "execution_count": 4,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m41.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m58.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h"
]
}
+ ],
+ "source": [
+ "!pip --quiet install faiss-cpu tiktoken"
]
},
{
"cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "eqcorKP4YEH6"
+ },
+ "outputs": [],
"source": [
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
"from langchain_community.document_loaders import WebBaseLoader\n",
@@ -206,15 +197,15 @@
")\n",
"\n",
"vectorstore_retriever = vectorstore.as_retriever()\n"
- ],
- "metadata": {
- "id": "eqcorKP4YEH6"
- },
- "execution_count": 5,
- "outputs": []
+ ]
},
{
"cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "eQFNJ-38abRd"
+ },
+ "outputs": [],
"source": [
"from langchain.tools.retriever import create_retriever_tool\n",
"\n",
@@ -223,25 +214,25 @@
" name=\"vectorstore_search\",\n",
" description=\"Retrieve relevant info from a vectorstore that contains information from Paul Graham about how to write good essays.\"\n",
")"
- ],
- "metadata": {
- "id": "eQFNJ-38abRd"
- },
- "execution_count": 6,
- "outputs": []
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "aB90i9W5YqWi"
+ },
"source": [
"#### Python interpreter tool\n",
"You can easily equip your agent with a python interpreter!"
- ],
- "metadata": {
- "id": "aB90i9W5YqWi"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "id": "AN-4FqXKYEFw"
+ },
+ "outputs": [],
"source": [
"from langchain.agents import Tool\n",
"from langchain_experimental.utilities import PythonREPL\n",
@@ -258,34 +249,34 @@
"class ToolInput(BaseModel):\n",
" code: str = Field(description=\"Python code to execute.\")\n",
"python_tool.args_schema = ToolInput"
- ],
- "metadata": {
- "id": "AN-4FqXKYEFw"
- },
- "execution_count": 7,
- "outputs": []
+ ]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": 7,
"metadata": {
"id": "8vj868w_YED_"
},
- "execution_count": 7,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "2SDCaoypexGL"
+ },
"source": [
"#### Transform any Python function in a Tool\n",
"You can easily equip your agent with any Python function!"
- ],
- "metadata": {
- "id": "2SDCaoypexGL"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "id": "DYUJT2xKewny"
+ },
+ "outputs": [],
"source": [
"from langchain_core.tools import tool\n",
"import random\n",
@@ -307,27 +298,27 @@
" a: int = Field(description=\"First input\")\n",
" b: int = Field(description=\"Second input\")\n",
"random_operation_tool.args_schema = random_operation_inputs"
- ],
- "metadata": {
- "id": "DYUJT2xKewny"
- },
- "execution_count": 8,
- "outputs": []
+ ]
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "hcspRBsRY2ED"
+ },
"source": [
"## Create ReAct Agent\n",
"\n",
"The model can smartly pick the right tool(s) for the user query, call them in any sequence, analyze the results and self-reflect. \n",
"Once the model considers it has enough information to answer the user question, it generates the final answer."
- ],
- "metadata": {
- "id": "hcspRBsRY2ED"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "id": "CY83aAprYECO"
+ },
+ "outputs": [],
"source": [
"from langchain.agents import AgentExecutor\n",
"from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n",
@@ -355,44 +346,30 @@
")\n",
"\n",
"agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool, random_operation_tool], verbose=True)"
- ],
- "metadata": {
- "id": "CY83aAprYECO"
- },
- "execution_count": 13,
- "outputs": []
+ ]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": 13,
"metadata": {
"id": "RK0NaqngYD_J"
},
- "execution_count": 13,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "mzTQcVXSaBe7"
+ },
"source": [
"## Ask a standalone question to the ReAct agent\n",
"A question that requires using a predefined tool from Langchain"
- ],
- "metadata": {
- "id": "mzTQcVXSaBe7"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the model smartly looks in the vector db, and then online"
- ],
+ "execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -401,11 +378,10 @@
"id": "rR2hWVp-aEsU",
"outputId": "fcf9b8b3-7027-4679-c06d-ef166f016892"
},
- "execution_count": 14,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\n",
"\n",
@@ -580,50 +556,51 @@
]
},
{
- "output_type": "execute_result",
"data": {
- "text/plain": [
- "\"Here are some tips for writing an essay:\\n- Start with a question that spurs some response.\\n- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\\n- You don't need a complete thesis, just a gap to explore.\\n- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\\n- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\\n- You can get breadth by reading and talking about a wide range of topics.\\n- You can get depth by doing and having to solve problems.\\n- You can also get ideas by talking to people who make you have new ideas.\\n\\nHere are some fun facts about the Roman Empire:\\n- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\\n- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\\n- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\""
- ],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
- }
+ },
+ "text/plain": [
+ "\"Here are some tips for writing an essay:\\n- Start with a question that spurs some response.\\n- Don't choose a topic at random, make sure you have a way in, a new insight or approach.\\n- You don't need a complete thesis, just a gap to explore.\\n- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.\\n- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.\\n- You can get breadth by reading and talking about a wide range of topics.\\n- You can get depth by doing and having to solve problems.\\n- You can also get ideas by talking to people who make you have new ideas.\\n\\nHere are some fun facts about the Roman Empire:\\n- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.\\n- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.\\n- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.\""
+ ]
},
+ "execution_count": 14,
"metadata": {},
- "execution_count": 14
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model smartly looks in the vector db, and then online"
]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": 14,
"metadata": {
"id": "DIP1YkXCg7rQ"
},
- "execution_count": 14,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
- "source": [
- "A question that requires the large language model to use a custom tool."
- ],
"metadata": {
"id": "FhwS3VHvg3_l"
- }
+ },
+ "source": [
+ "A question that requires the large language model to use a custom tool."
+ ]
},
{
"cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"Calculate the result of the random operation of 10 and 20. Then find a few fun facts about that number, as well as its prime factors.\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the model uses a sequence of tools"
- ],
+ "execution_count": 22,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -632,11 +609,10 @@
"id": "79Dw6Zhrg3xI",
"outputId": "a41c6de9-cb64-4cff-a58b-e9481d438f6c"
},
- "execution_count": 22,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\n",
"\n",
@@ -672,50 +648,51 @@
]
},
{
- "output_type": "execute_result",
"data": {
- "text/plain": [
- "\"The result of the random operation is **200**. Here are some fun facts about the number 200:\\n- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\\n- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\\n- The number 200 is an even composite number composed of two distinct primes.\\n\\nThe prime factors of 200 are 2 and 5.\""
- ],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
- }
+ },
+ "text/plain": [
+ "\"The result of the random operation is **200**. Here are some fun facts about the number 200:\\n- It is the smallest base 10 unprimeable number, meaning it cannot be turned into a prime number by changing just one of its digits to any other digit.\\n- According to Bullinger's study of biblical literature, the number 200 signifies 'insufficiency'.\\n- The number 200 is an even composite number composed of two distinct primes.\\n\\nThe prime factors of 200 are 2 and 5.\""
+ ]
},
+ "execution_count": 22,
"metadata": {},
- "execution_count": 22
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Calculate the result of the random operation of 10 and 20. Then find a few fun facts about that number, as well as its prime factors.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the model uses a sequence of tools"
]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": 15,
"metadata": {
"id": "V8LcAh8vaEqR"
},
- "execution_count": 15,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
- "source": [
- "A question that requires the large language model to directly answer."
- ],
"metadata": {
"id": "n9nV_jiaAaD1"
- }
+ },
+ "source": [
+ "A question that requires the large language model to directly answer."
+ ]
},
{
"cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"Hey how are you?\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']\n",
- "\n",
- "# note that the modle can directly answer!"
- ],
+ "execution_count": 23,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -724,11 +701,10 @@
"id": "tRf6V3gJAZkO",
"outputId": "aec1acdb-876c-48e9-a1f5-e3ba8a0f19ed"
},
- "execution_count": 23,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\n",
"\n",
@@ -745,49 +721,52 @@
]
},
{
- "output_type": "execute_result",
"data": {
- "text/plain": [
- "\"I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\""
- ],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
- }
+ },
+ "text/plain": [
+ "\"I'm an AI chatbot, so I don't have feelings as such, but I'm here to help you with your queries. How can I help?\""
+ ]
},
+ "execution_count": 23,
"metadata": {},
- "execution_count": 23
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"Hey how are you?\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']\n",
+ "\n",
+ "# note that the modle can directly answer!"
]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": null,
"metadata": {
"id": "hnLkln_ckYXJ"
},
- "execution_count": null,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "3h-icRL2_5iu"
+ },
"source": [
"## Ask a more complex question to the ReAct agent\n",
"A question that requires using multipe tools, in sequence"
- ],
- "metadata": {
- "id": "3h-icRL2_5iu"
- }
+ ]
},
{
"cell_type": "code",
- "source": [
- "response = agent_executor.invoke({\n",
- " \"input\": \"In what year was the company that was founded as Sound of Music went public? What was its stock price in 2000 and 2010.\",\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']"
- ],
+ "execution_count": 27,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -796,11 +775,10 @@
"id": "Q4IyoXXRaEoL",
"outputId": "0e935f68-8b7c-4d9a-b8f1-1be57b7c9a1f"
},
- "execution_count": 27,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\n",
"\n",
@@ -822,41 +800,54 @@
]
},
{
- "output_type": "execute_result",
"data": {
- "text/plain": [
- "'Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.'"
- ],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
- }
+ },
+ "text/plain": [
+ "'Best Buy, which was founded as Sound of Music, went public in 1985. Its stock price in 2000 was between $5.93 and $16.24 and in 2010, it was between $20.27 and $31.24.'"
+ ]
},
+ "execution_count": 27,
"metadata": {},
- "execution_count": 27
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "response = agent_executor.invoke({\n",
+ " \"input\": \"In what year was the company that was founded as Sound of Music went public? What was its stock price in 2000 and 2010.\",\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": null,
"metadata": {
"id": "L1BsKueTaEmg"
},
- "execution_count": null,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "markdown",
+ "metadata": {
+ "id": "53FDSFzQBFyf"
+ },
"source": [
"## Have a multi-turn conversation with the ReAct agent\n",
"The chat history enables you to have multi-turn conversations with the ReAct agent."
- ],
- "metadata": {
- "id": "53FDSFzQBFyf"
- }
+ ]
},
{
"cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "id": "1iYyAExsaEk9"
+ },
+ "outputs": [],
"source": [
"# Step 1: Construct the chat history as a list of LangChain Messages, ending with the last user message\n",
"from langchain_core.messages import HumanMessage, AIMessage\n",
@@ -868,15 +859,15 @@
"]\n",
"\n",
"prompt = ChatPromptTemplate.from_messages(chat_history)"
- ],
- "metadata": {
- "id": "1iYyAExsaEk9"
- },
- "execution_count": 28,
- "outputs": []
+ ]
},
{
"cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "id": "fyv_1LedaEi7"
+ },
+ "outputs": [],
"source": [
"# Step 2: When you make the agent, specify the chat_history as the prompt, e.g.\n",
"# Create the ReAct agent\n",
@@ -887,36 +878,23 @@
")\n",
"\n",
"agent_executor = AgentExecutor(agent=agent, tools=[internet_search, vectorstore_search, python_tool], verbose=True)"
- ],
- "metadata": {
- "id": "fyv_1LedaEi7"
- },
- "execution_count": 29,
- "outputs": []
+ ]
},
{
"cell_type": "code",
- "source": [
- "# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke\n",
- "response = agent_executor.invoke({\n",
- " \"preamble\": preamble,\n",
- "})\n",
- "\n",
- "response['output']"
- ],
+ "execution_count": 30,
"metadata": {
- "id": "-ZCFj-m5nqFw",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 538
},
+ "id": "-ZCFj-m5nqFw",
"outputId": "3d71931e-1644-49b9-903c-4ff117787868"
},
- "execution_count": 30,
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"\n",
"\n",
@@ -943,37 +921,59 @@
]
},
{
- "output_type": "execute_result",
"data": {
- "text/plain": [
- "\"Oracle's CRM offering includes the following:\\n- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\\n- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\\n- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\\n- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\\n- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\""
- ],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
- }
+ },
+ "text/plain": [
+ "\"Oracle's CRM offering includes the following:\\n- Marketing Cloud, which provides a comprehensive set of tools that cover a range of digital marketing needs, including cross-channel marketing, marketing automation, data management, content marketing and social media marketing\\n- CRM On Demand, which is a full-featured marketing automation product with features from lead management and nurturing, to measuring marketing ROI\\n- Siebel CRM, which delivers a combination of transactional, analytical, and engagement features to manage all customer-facing operations, with solutions tailored to more than 20 industries\\n- Siebel CRM Life Sciences, which provides personalised content delivery tools to help sales and marketing teams deliver the right messages during each customer interaction\\n- Oracle Client Experience (CX), a connected suite of tools that transcends standard CRM\""
+ ]
},
+ "execution_count": 30,
"metadata": {},
- "execution_count": 30
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke\n",
+ "response = agent_executor.invoke({\n",
+ " \"preamble\": preamble,\n",
+ "})\n",
+ "\n",
+ "response['output']"
]
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": 30,
"metadata": {
"id": "8Zh01M4hWw3L"
},
- "execution_count": 30,
- "outputs": []
+ "outputs": [],
+ "source": []
},
{
"cell_type": "code",
- "source": [],
+ "execution_count": null,
"metadata": {
"id": "0qEPY6xj_qwr"
},
- "execution_count": null,
- "outputs": []
+ "outputs": [],
+ "source": []
}
- ]
-}
\ No newline at end of file
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/poetry.lock b/poetry.lock
index 48492ac19..8d9cf7a23 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -31,6 +31,46 @@ doc = ["Sphinx (>=7.4,<8.0)", "packaging", "sphinx-autodoc-typehints (>=1.2.0)",
test = ["anyio[trio]", "coverage[toml] (>=7)", "exceptiongroup (>=1.2.0)", "hypothesis (>=4.0)", "psutil (>=5.9)", "pytest (>=7.0)", "pytest-mock (>=3.6.1)", "trustme", "truststore (>=0.9.1)", "uvloop (>=0.21.0b1)"]
trio = ["trio (>=0.26.1)"]
+[[package]]
+name = "attrs"
+version = "24.2.0"
+description = "Classes Without Boilerplate"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "attrs-24.2.0-py3-none-any.whl", hash = "sha256:81921eb96de3191c8258c199618104dd27ac608d9366f5e35d011eae1867ede2"},
+ {file = "attrs-24.2.0.tar.gz", hash = "sha256:5cfb1b9148b5b086569baec03f20d7b6bf3bcacc9a42bebf87ffaaca362f6346"},
+]
+
+[package.extras]
+benchmark = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-codspeed", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
+cov = ["cloudpickle", "coverage[toml] (>=5.3)", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
+dev = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pre-commit", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
+docs = ["cogapp", "furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier (<24.7)"]
+tests = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
+tests-mypy = ["mypy (>=1.11.1)", "pytest-mypy-plugins"]
+
+[[package]]
+name = "beautifulsoup4"
+version = "4.12.3"
+description = "Screen-scraping library"
+optional = false
+python-versions = ">=3.6.0"
+files = [
+ {file = "beautifulsoup4-4.12.3-py3-none-any.whl", hash = "sha256:b80878c9f40111313e55da8ba20bdba06d8fa3969fc68304167741bbf9e082ed"},
+ {file = "beautifulsoup4-4.12.3.tar.gz", hash = "sha256:74e3d1928edc070d21748185c46e3fb33490f22f52a3addee9aee0f4f7781051"},
+]
+
+[package.dependencies]
+soupsieve = ">1.2"
+
+[package.extras]
+cchardet = ["cchardet"]
+chardet = ["chardet"]
+charset-normalizer = ["charset-normalizer"]
+html5lib = ["html5lib"]
+lxml = ["lxml"]
+
[[package]]
name = "black"
version = "24.10.0"
@@ -75,6 +115,23 @@ d = ["aiohttp (>=3.10)"]
jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
uvloop = ["uvloop (>=0.15.2)"]
+[[package]]
+name = "bleach"
+version = "6.2.0"
+description = "An easy safelist-based HTML-sanitizing tool."
+optional = false
+python-versions = ">=3.9"
+files = [
+ {file = "bleach-6.2.0-py3-none-any.whl", hash = "sha256:117d9c6097a7c3d22fd578fcd8d35ff1e125df6736f554da4e432fdd63f31e5e"},
+ {file = "bleach-6.2.0.tar.gz", hash = "sha256:123e894118b8a599fd80d3ec1a6d4cc7ce4e5882b1317a7e1ba69b56e95f991f"},
+]
+
+[package.dependencies]
+webencodings = "*"
+
+[package.extras]
+css = ["tinycss2 (>=1.1.0,<1.5)"]
+
[[package]]
name = "certifi"
version = "2024.8.30"
@@ -86,6 +143,85 @@ files = [
{file = "certifi-2024.8.30.tar.gz", hash = "sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9"},
]
+[[package]]
+name = "cffi"
+version = "1.17.1"
+description = "Foreign Function Interface for Python calling C code."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "cffi-1.17.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14"},
+ {file = "cffi-1.17.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67"},
+ {file = "cffi-1.17.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382"},
+ {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702"},
+ {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3"},
+ {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6"},
+ {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17"},
+ {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8"},
+ {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e"},
+ {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be"},
+ {file = "cffi-1.17.1-cp310-cp310-win32.whl", hash = "sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c"},
+ {file = "cffi-1.17.1-cp310-cp310-win_amd64.whl", hash = "sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15"},
+ {file = "cffi-1.17.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401"},
+ {file = "cffi-1.17.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf"},
+ {file = "cffi-1.17.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4"},
+ {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41"},
+ {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1"},
+ {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6"},
+ {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d"},
+ {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6"},
+ {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f"},
+ {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b"},
+ {file = "cffi-1.17.1-cp311-cp311-win32.whl", hash = "sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655"},
+ {file = "cffi-1.17.1-cp311-cp311-win_amd64.whl", hash = "sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0"},
+ {file = "cffi-1.17.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4"},
+ {file = "cffi-1.17.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c"},
+ {file = "cffi-1.17.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36"},
+ {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5"},
+ {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff"},
+ {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99"},
+ {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93"},
+ {file = "cffi-1.17.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3"},
+ {file = "cffi-1.17.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8"},
+ {file = "cffi-1.17.1-cp312-cp312-win32.whl", hash = "sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65"},
+ {file = "cffi-1.17.1-cp312-cp312-win_amd64.whl", hash = "sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903"},
+ {file = "cffi-1.17.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e"},
+ {file = "cffi-1.17.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2"},
+ {file = "cffi-1.17.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3"},
+ {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683"},
+ {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5"},
+ {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4"},
+ {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd"},
+ {file = "cffi-1.17.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed"},
+ {file = "cffi-1.17.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9"},
+ {file = "cffi-1.17.1-cp313-cp313-win32.whl", hash = "sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d"},
+ {file = "cffi-1.17.1-cp313-cp313-win_amd64.whl", hash = "sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a"},
+ {file = "cffi-1.17.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b"},
+ {file = "cffi-1.17.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964"},
+ {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9"},
+ {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc"},
+ {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c"},
+ {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1"},
+ {file = "cffi-1.17.1-cp38-cp38-win32.whl", hash = "sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8"},
+ {file = "cffi-1.17.1-cp38-cp38-win_amd64.whl", hash = "sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1"},
+ {file = "cffi-1.17.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16"},
+ {file = "cffi-1.17.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36"},
+ {file = "cffi-1.17.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8"},
+ {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576"},
+ {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87"},
+ {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0"},
+ {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3"},
+ {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595"},
+ {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a"},
+ {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e"},
+ {file = "cffi-1.17.1-cp39-cp39-win32.whl", hash = "sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7"},
+ {file = "cffi-1.17.1-cp39-cp39-win_amd64.whl", hash = "sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662"},
+ {file = "cffi-1.17.1.tar.gz", hash = "sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824"},
+]
+
+[package.dependencies]
+pycparser = "*"
+
[[package]]
name = "charset-normalizer"
version = "3.4.0"
@@ -251,6 +387,17 @@ files = [
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
]
+[[package]]
+name = "defusedxml"
+version = "0.7.1"
+description = "XML bomb protection for Python stdlib modules"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+files = [
+ {file = "defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61"},
+ {file = "defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69"},
+]
+
[[package]]
name = "fastavro"
version = "1.9.7"
@@ -297,6 +444,20 @@ lz4 = ["lz4"]
snappy = ["cramjam"]
zstandard = ["zstandard"]
+[[package]]
+name = "fastjsonschema"
+version = "2.21.1"
+description = "Fastest Python implementation of JSON schema"
+optional = false
+python-versions = "*"
+files = [
+ {file = "fastjsonschema-2.21.1-py3-none-any.whl", hash = "sha256:c9e5b7e908310918cf494a434eeb31384dd84a98b57a30bcb1f535015b554667"},
+ {file = "fastjsonschema-2.21.1.tar.gz", hash = "sha256:794d4f0a58f848961ba16af7b9c85a3e88cd360df008c59aac6fc5ae9323b5d4"},
+]
+
+[package.extras]
+devel = ["colorama", "json-spec", "jsonschema", "pylint", "pytest", "pytest-benchmark", "pytest-cache", "validictory"]
+
[[package]]
name = "filelock"
version = "3.16.1"
@@ -468,6 +629,250 @@ files = [
[package.extras]
all = ["flake8 (>=7.1.1)", "mypy (>=1.11.2)", "pytest (>=8.3.2)", "ruff (>=0.6.2)"]
+[[package]]
+name = "jinja2"
+version = "3.1.4"
+description = "A very fast and expressive template engine."
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "jinja2-3.1.4-py3-none-any.whl", hash = "sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d"},
+ {file = "jinja2-3.1.4.tar.gz", hash = "sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369"},
+]
+
+[package.dependencies]
+MarkupSafe = ">=2.0"
+
+[package.extras]
+i18n = ["Babel (>=2.7)"]
+
+[[package]]
+name = "jsonschema"
+version = "4.23.0"
+description = "An implementation of JSON Schema validation for Python"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "jsonschema-4.23.0-py3-none-any.whl", hash = "sha256:fbadb6f8b144a8f8cf9f0b89ba94501d143e50411a1278633f56a7acf7fd5566"},
+ {file = "jsonschema-4.23.0.tar.gz", hash = "sha256:d71497fef26351a33265337fa77ffeb82423f3ea21283cd9467bb03999266bc4"},
+]
+
+[package.dependencies]
+attrs = ">=22.2.0"
+jsonschema-specifications = ">=2023.03.6"
+referencing = ">=0.28.4"
+rpds-py = ">=0.7.1"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=24.6.0)"]
+
+[[package]]
+name = "jsonschema-specifications"
+version = "2024.10.1"
+description = "The JSON Schema meta-schemas and vocabularies, exposed as a Registry"
+optional = false
+python-versions = ">=3.9"
+files = [
+ {file = "jsonschema_specifications-2024.10.1-py3-none-any.whl", hash = "sha256:a09a0680616357d9a0ecf05c12ad234479f549239d0f5b55f3deea67475da9bf"},
+ {file = "jsonschema_specifications-2024.10.1.tar.gz", hash = "sha256:0f38b83639958ce1152d02a7f062902c41c8fd20d558b0c34344292d417ae272"},
+]
+
+[package.dependencies]
+referencing = ">=0.31.0"
+
+[[package]]
+name = "jupyter-client"
+version = "8.6.3"
+description = "Jupyter protocol implementation and client libraries"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "jupyter_client-8.6.3-py3-none-any.whl", hash = "sha256:e8a19cc986cc45905ac3362915f410f3af85424b4c0905e94fa5f2cb08e8f23f"},
+ {file = "jupyter_client-8.6.3.tar.gz", hash = "sha256:35b3a0947c4a6e9d589eb97d7d4cd5e90f910ee73101611f01283732bd6d9419"},
+]
+
+[package.dependencies]
+jupyter-core = ">=4.12,<5.0.dev0 || >=5.1.dev0"
+python-dateutil = ">=2.8.2"
+pyzmq = ">=23.0"
+tornado = ">=6.2"
+traitlets = ">=5.3"
+
+[package.extras]
+docs = ["ipykernel", "myst-parser", "pydata-sphinx-theme", "sphinx (>=4)", "sphinx-autodoc-typehints", "sphinxcontrib-github-alt", "sphinxcontrib-spelling"]
+test = ["coverage", "ipykernel (>=6.14)", "mypy", "paramiko", "pre-commit", "pytest (<8.2.0)", "pytest-cov", "pytest-jupyter[client] (>=0.4.1)", "pytest-timeout"]
+
+[[package]]
+name = "jupyter-core"
+version = "5.7.2"
+description = "Jupyter core package. A base package on which Jupyter projects rely."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "jupyter_core-5.7.2-py3-none-any.whl", hash = "sha256:4f7315d2f6b4bcf2e3e7cb6e46772eba760ae459cd1f59d29eb57b0a01bd7409"},
+ {file = "jupyter_core-5.7.2.tar.gz", hash = "sha256:aa5f8d32bbf6b431ac830496da7392035d6f61b4f54872f15c4bd2a9c3f536d9"},
+]
+
+[package.dependencies]
+platformdirs = ">=2.5"
+pywin32 = {version = ">=300", markers = "sys_platform == \"win32\" and platform_python_implementation != \"PyPy\""}
+traitlets = ">=5.3"
+
+[package.extras]
+docs = ["myst-parser", "pydata-sphinx-theme", "sphinx-autodoc-typehints", "sphinxcontrib-github-alt", "sphinxcontrib-spelling", "traitlets"]
+test = ["ipykernel", "pre-commit", "pytest (<8)", "pytest-cov", "pytest-timeout"]
+
+[[package]]
+name = "jupyterlab-pygments"
+version = "0.3.0"
+description = "Pygments theme using JupyterLab CSS variables"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "jupyterlab_pygments-0.3.0-py3-none-any.whl", hash = "sha256:841a89020971da1d8693f1a99997aefc5dc424bb1b251fd6322462a1b8842780"},
+ {file = "jupyterlab_pygments-0.3.0.tar.gz", hash = "sha256:721aca4d9029252b11cfa9d185e5b5af4d54772bb8072f9b7036f4170054d35d"},
+]
+
+[[package]]
+name = "markupsafe"
+version = "3.0.2"
+description = "Safely add untrusted strings to HTML/XML markup."
+optional = false
+python-versions = ">=3.9"
+files = [
+ {file = "MarkupSafe-3.0.2-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:7e94c425039cde14257288fd61dcfb01963e658efbc0ff54f5306b06054700f8"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9e2d922824181480953426608b81967de705c3cef4d1af983af849d7bd619158"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:38a9ef736c01fccdd6600705b09dc574584b89bea478200c5fbf112a6b0d5579"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bbcb445fa71794da8f178f0f6d66789a28d7319071af7a496d4d507ed566270d"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:57cb5a3cf367aeb1d316576250f65edec5bb3be939e9247ae594b4bcbc317dfb"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:3809ede931876f5b2ec92eef964286840ed3540dadf803dd570c3b7e13141a3b"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:e07c3764494e3776c602c1e78e298937c3315ccc9043ead7e685b7f2b8d47b3c"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:b424c77b206d63d500bcb69fa55ed8d0e6a3774056bdc4839fc9298a7edca171"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-win32.whl", hash = "sha256:fcabf5ff6eea076f859677f5f0b6b5c1a51e70a376b0579e0eadef8db48c6b50"},
+ {file = "MarkupSafe-3.0.2-cp310-cp310-win_amd64.whl", hash = "sha256:6af100e168aa82a50e186c82875a5893c5597a0c1ccdb0d8b40240b1f28b969a"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:9025b4018f3a1314059769c7bf15441064b2207cb3f065e6ea1e7359cb46db9d"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:93335ca3812df2f366e80509ae119189886b0f3c2b81325d39efdb84a1e2ae93"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2cb8438c3cbb25e220c2ab33bb226559e7afb3baec11c4f218ffa7308603c832"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a123e330ef0853c6e822384873bef7507557d8e4a082961e1defa947aa59ba84"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1e084f686b92e5b83186b07e8a17fc09e38fff551f3602b249881fec658d3eca"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d8213e09c917a951de9d09ecee036d5c7d36cb6cb7dbaece4c71a60d79fb9798"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:5b02fb34468b6aaa40dfc198d813a641e3a63b98c2b05a16b9f80b7ec314185e"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:0bff5e0ae4ef2e1ae4fdf2dfd5b76c75e5c2fa4132d05fc1b0dabcd20c7e28c4"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-win32.whl", hash = "sha256:6c89876f41da747c8d3677a2b540fb32ef5715f97b66eeb0c6b66f5e3ef6f59d"},
+ {file = "MarkupSafe-3.0.2-cp311-cp311-win_amd64.whl", hash = "sha256:70a87b411535ccad5ef2f1df5136506a10775d267e197e4cf531ced10537bd6b"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:9778bd8ab0a994ebf6f84c2b949e65736d5575320a17ae8984a77fab08db94cf"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:846ade7b71e3536c4e56b386c2a47adf5741d2d8b94ec9dc3e92e5e1ee1e2225"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1c99d261bd2d5f6b59325c92c73df481e05e57f19837bdca8413b9eac4bd8028"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e17c96c14e19278594aa4841ec148115f9c7615a47382ecb6b82bd8fea3ab0c8"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:88416bd1e65dcea10bc7569faacb2c20ce071dd1f87539ca2ab364bf6231393c"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2181e67807fc2fa785d0592dc2d6206c019b9502410671cc905d132a92866557"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:52305740fe773d09cffb16f8ed0427942901f00adedac82ec8b67752f58a1b22"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ad10d3ded218f1039f11a75f8091880239651b52e9bb592ca27de44eed242a48"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-win32.whl", hash = "sha256:0f4ca02bea9a23221c0182836703cbf8930c5e9454bacce27e767509fa286a30"},
+ {file = "MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl", hash = "sha256:8e06879fc22a25ca47312fbe7c8264eb0b662f6db27cb2d3bbbc74b1df4b9b87"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:ba9527cdd4c926ed0760bc301f6728ef34d841f405abf9d4f959c478421e4efd"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f8b3d067f2e40fe93e1ccdd6b2e1d16c43140e76f02fb1319a05cf2b79d99430"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:569511d3b58c8791ab4c2e1285575265991e6d8f8700c7be0e88f86cb0672094"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:15ab75ef81add55874e7ab7055e9c397312385bd9ced94920f2802310c930396"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f3818cb119498c0678015754eba762e0d61e5b52d34c8b13d770f0719f7b1d79"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:cdb82a876c47801bb54a690c5ae105a46b392ac6099881cdfb9f6e95e4014c6a"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:cabc348d87e913db6ab4aa100f01b08f481097838bdddf7c7a84b7575b7309ca"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:444dcda765c8a838eaae23112db52f1efaf750daddb2d9ca300bcae1039adc5c"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-win32.whl", hash = "sha256:bcf3e58998965654fdaff38e58584d8937aa3096ab5354d493c77d1fdd66d7a1"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:e6a2a455bd412959b57a172ce6328d2dd1f01cb2135efda2e4576e8a23fa3b0f"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:b5a6b3ada725cea8a5e634536b1b01c30bcdcd7f9c6fff4151548d5bf6b3a36c"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:a904af0a6162c73e3edcb969eeeb53a63ceeb5d8cf642fade7d39e7963a22ddb"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4aa4e5faecf353ed117801a068ebab7b7e09ffb6e1d5e412dc852e0da018126c"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c0ef13eaeee5b615fb07c9a7dadb38eac06a0608b41570d8ade51c56539e509d"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d16a81a06776313e817c951135cf7340a3e91e8c1ff2fac444cfd75fffa04afe"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:6381026f158fdb7c72a168278597a5e3a5222e83ea18f543112b2662a9b699c5"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:3d79d162e7be8f996986c064d1c7c817f6df3a77fe3d6859f6f9e7be4b8c213a"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:131a3c7689c85f5ad20f9f6fb1b866f402c445b220c19fe4308c0b147ccd2ad9"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-win32.whl", hash = "sha256:ba8062ed2cf21c07a9e295d5b8a2a5ce678b913b45fdf68c32d95d6c1291e0b6"},
+ {file = "MarkupSafe-3.0.2-cp313-cp313t-win_amd64.whl", hash = "sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:eaa0a10b7f72326f1372a713e73c3f739b524b3af41feb43e4921cb529f5929a"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:48032821bbdf20f5799ff537c7ac3d1fba0ba032cfc06194faffa8cda8b560ff"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1a9d3f5f0901fdec14d8d2f66ef7d035f2157240a433441719ac9a3fba440b13"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:88b49a3b9ff31e19998750c38e030fc7bb937398b1f78cfa599aaef92d693144"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cfad01eed2c2e0c01fd0ecd2ef42c492f7f93902e39a42fc9ee1692961443a29"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:1225beacc926f536dc82e45f8a4d68502949dc67eea90eab715dea3a21c1b5f0"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:3169b1eefae027567d1ce6ee7cae382c57fe26e82775f460f0b2778beaad66c0"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:eb7972a85c54febfb25b5c4b4f3af4dcc731994c7da0d8a0b4a6eb0640e1d178"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-win32.whl", hash = "sha256:8c4e8c3ce11e1f92f6536ff07154f9d49677ebaaafc32db9db4620bc11ed480f"},
+ {file = "MarkupSafe-3.0.2-cp39-cp39-win_amd64.whl", hash = "sha256:6e296a513ca3d94054c2c881cc913116e90fd030ad1c656b3869762b754f5f8a"},
+ {file = "markupsafe-3.0.2.tar.gz", hash = "sha256:ee55d3edf80167e48ea11a923c7386f4669df67d7994554387f84e7d8b0a2bf0"},
+]
+
+[[package]]
+name = "mistune"
+version = "3.0.2"
+description = "A sane and fast Markdown parser with useful plugins and renderers"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "mistune-3.0.2-py3-none-any.whl", hash = "sha256:71481854c30fdbc938963d3605b72501f5c10a9320ecd412c121c163a1c7d205"},
+ {file = "mistune-3.0.2.tar.gz", hash = "sha256:fc7f93ded930c92394ef2cb6f04a8aabab4117a91449e72dcc8dfa646a508be8"},
+]
+
+[[package]]
+name = "mypy"
+version = "1.14.1"
+description = "Optional static typing for Python"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "mypy-1.14.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:52686e37cf13d559f668aa398dd7ddf1f92c5d613e4f8cb262be2fb4fedb0fcb"},
+ {file = "mypy-1.14.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:1fb545ca340537d4b45d3eecdb3def05e913299ca72c290326be19b3804b39c0"},
+ {file = "mypy-1.14.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:90716d8b2d1f4cd503309788e51366f07c56635a3309b0f6a32547eaaa36a64d"},
+ {file = "mypy-1.14.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2ae753f5c9fef278bcf12e1a564351764f2a6da579d4a81347e1d5a15819997b"},
+ {file = "mypy-1.14.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:e0fe0f5feaafcb04505bcf439e991c6d8f1bf8b15f12b05feeed96e9e7bf1427"},
+ {file = "mypy-1.14.1-cp310-cp310-win_amd64.whl", hash = "sha256:7d54bd85b925e501c555a3227f3ec0cfc54ee8b6930bd6141ec872d1c572f81f"},
+ {file = "mypy-1.14.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f995e511de847791c3b11ed90084a7a0aafdc074ab88c5a9711622fe4751138c"},
+ {file = "mypy-1.14.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d64169ec3b8461311f8ce2fd2eb5d33e2d0f2c7b49116259c51d0d96edee48d1"},
+ {file = "mypy-1.14.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ba24549de7b89b6381b91fbc068d798192b1b5201987070319889e93038967a8"},
+ {file = "mypy-1.14.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:183cf0a45457d28ff9d758730cd0210419ac27d4d3f285beda038c9083363b1f"},
+ {file = "mypy-1.14.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f2a0ecc86378f45347f586e4163d1769dd81c5a223d577fe351f26b179e148b1"},
+ {file = "mypy-1.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:ad3301ebebec9e8ee7135d8e3109ca76c23752bac1e717bc84cd3836b4bf3eae"},
+ {file = "mypy-1.14.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:30ff5ef8519bbc2e18b3b54521ec319513a26f1bba19a7582e7b1f58a6e69f14"},
+ {file = "mypy-1.14.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:cb9f255c18052343c70234907e2e532bc7e55a62565d64536dbc7706a20b78b9"},
+ {file = "mypy-1.14.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8b4e3413e0bddea671012b063e27591b953d653209e7a4fa5e48759cda77ca11"},
+ {file = "mypy-1.14.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:553c293b1fbdebb6c3c4030589dab9fafb6dfa768995a453d8a5d3b23784af2e"},
+ {file = "mypy-1.14.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fad79bfe3b65fe6a1efaed97b445c3d37f7be9fdc348bdb2d7cac75579607c89"},
+ {file = "mypy-1.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:8fa2220e54d2946e94ab6dbb3ba0a992795bd68b16dc852db33028df2b00191b"},
+ {file = "mypy-1.14.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:92c3ed5afb06c3a8e188cb5da4984cab9ec9a77ba956ee419c68a388b4595255"},
+ {file = "mypy-1.14.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:dbec574648b3e25f43d23577309b16534431db4ddc09fda50841f1e34e64ed34"},
+ {file = "mypy-1.14.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8c6d94b16d62eb3e947281aa7347d78236688e21081f11de976376cf010eb31a"},
+ {file = "mypy-1.14.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d4b19b03fdf54f3c5b2fa474c56b4c13c9dbfb9a2db4370ede7ec11a2c5927d9"},
+ {file = "mypy-1.14.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:0c911fde686394753fff899c409fd4e16e9b294c24bfd5e1ea4675deae1ac6fd"},
+ {file = "mypy-1.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:8b21525cb51671219f5307be85f7e646a153e5acc656e5cebf64bfa076c50107"},
+ {file = "mypy-1.14.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:7084fb8f1128c76cd9cf68fe5971b37072598e7c31b2f9f95586b65c741a9d31"},
+ {file = "mypy-1.14.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:8f845a00b4f420f693f870eaee5f3e2692fa84cc8514496114649cfa8fd5e2c6"},
+ {file = "mypy-1.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:44bf464499f0e3a2d14d58b54674dee25c031703b2ffc35064bd0df2e0fac319"},
+ {file = "mypy-1.14.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c99f27732c0b7dc847adb21c9d47ce57eb48fa33a17bc6d7d5c5e9f9e7ae5bac"},
+ {file = "mypy-1.14.1-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:bce23c7377b43602baa0bd22ea3265c49b9ff0b76eb315d6c34721af4cdf1d9b"},
+ {file = "mypy-1.14.1-cp38-cp38-win_amd64.whl", hash = "sha256:8edc07eeade7ebc771ff9cf6b211b9a7d93687ff892150cb5692e4f4272b0837"},
+ {file = "mypy-1.14.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:3888a1816d69f7ab92092f785a462944b3ca16d7c470d564165fe703b0970c35"},
+ {file = "mypy-1.14.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:46c756a444117c43ee984bd055db99e498bc613a70bbbc120272bd13ca579fbc"},
+ {file = "mypy-1.14.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:27fc248022907e72abfd8e22ab1f10e903915ff69961174784a3900a8cba9ad9"},
+ {file = "mypy-1.14.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:499d6a72fb7e5de92218db961f1a66d5f11783f9ae549d214617edab5d4dbdbb"},
+ {file = "mypy-1.14.1-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:57961db9795eb566dc1d1b4e9139ebc4c6b0cb6e7254ecde69d1552bf7613f60"},
+ {file = "mypy-1.14.1-cp39-cp39-win_amd64.whl", hash = "sha256:07ba89fdcc9451f2ebb02853deb6aaaa3d2239a236669a63ab3801bbf923ef5c"},
+ {file = "mypy-1.14.1-py3-none-any.whl", hash = "sha256:b66a60cc4073aeb8ae00057f9c1f64d49e90f918fbcef9a977eb121da8b8f1d1"},
+ {file = "mypy-1.14.1.tar.gz", hash = "sha256:7ec88144fe9b510e8475ec2f5f251992690fcf89ccb4500b214b4226abcd32d6"},
+]
+
+[package.dependencies]
+mypy_extensions = ">=1.0.0"
+typing_extensions = ">=4.6.0"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+faster-cache = ["orjson"]
+install-types = ["pip"]
+mypyc = ["setuptools (>=50)"]
+reports = ["lxml"]
+
[[package]]
name = "mypy-extensions"
version = "1.0.0"
@@ -479,6 +884,86 @@ files = [
{file = "mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"},
]
+[[package]]
+name = "nbclient"
+version = "0.10.1"
+description = "A client library for executing notebooks. Formerly nbconvert's ExecutePreprocessor."
+optional = false
+python-versions = ">=3.8.0"
+files = [
+ {file = "nbclient-0.10.1-py3-none-any.whl", hash = "sha256:949019b9240d66897e442888cfb618f69ef23dc71c01cb5fced8499c2cfc084d"},
+ {file = "nbclient-0.10.1.tar.gz", hash = "sha256:3e93e348ab27e712acd46fccd809139e356eb9a31aab641d1a7991a6eb4e6f68"},
+]
+
+[package.dependencies]
+jupyter-client = ">=6.1.12"
+jupyter-core = ">=4.12,<5.0.dev0 || >=5.1.dev0"
+nbformat = ">=5.1"
+traitlets = ">=5.4"
+
+[package.extras]
+dev = ["pre-commit"]
+docs = ["autodoc-traits", "flaky", "ipykernel (>=6.19.3)", "ipython", "ipywidgets", "mock", "moto", "myst-parser", "nbconvert (>=7.0.0)", "pytest (>=7.0,<8)", "pytest-asyncio", "pytest-cov (>=4.0)", "sphinx (>=1.7)", "sphinx-book-theme", "sphinxcontrib-spelling", "testpath", "xmltodict"]
+test = ["flaky", "ipykernel (>=6.19.3)", "ipython", "ipywidgets", "nbconvert (>=7.0.0)", "pytest (>=7.0,<8)", "pytest-asyncio", "pytest-cov (>=4.0)", "testpath", "xmltodict"]
+
+[[package]]
+name = "nbconvert"
+version = "7.16.4"
+description = "Converting Jupyter Notebooks (.ipynb files) to other formats. Output formats include asciidoc, html, latex, markdown, pdf, py, rst, script. nbconvert can be used both as a Python library (`import nbconvert`) or as a command line tool (invoked as `jupyter nbconvert ...`)."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "nbconvert-7.16.4-py3-none-any.whl", hash = "sha256:05873c620fe520b6322bf8a5ad562692343fe3452abda5765c7a34b7d1aa3eb3"},
+ {file = "nbconvert-7.16.4.tar.gz", hash = "sha256:86ca91ba266b0a448dc96fa6c5b9d98affabde2867b363258703536807f9f7f4"},
+]
+
+[package.dependencies]
+beautifulsoup4 = "*"
+bleach = "!=5.0.0"
+defusedxml = "*"
+jinja2 = ">=3.0"
+jupyter-core = ">=4.7"
+jupyterlab-pygments = "*"
+markupsafe = ">=2.0"
+mistune = ">=2.0.3,<4"
+nbclient = ">=0.5.0"
+nbformat = ">=5.7"
+packaging = "*"
+pandocfilters = ">=1.4.1"
+pygments = ">=2.4.1"
+tinycss2 = "*"
+traitlets = ">=5.1"
+
+[package.extras]
+all = ["flaky", "ipykernel", "ipython", "ipywidgets (>=7.5)", "myst-parser", "nbsphinx (>=0.2.12)", "playwright", "pydata-sphinx-theme", "pyqtwebengine (>=5.15)", "pytest (>=7)", "sphinx (==5.0.2)", "sphinxcontrib-spelling", "tornado (>=6.1)"]
+docs = ["ipykernel", "ipython", "myst-parser", "nbsphinx (>=0.2.12)", "pydata-sphinx-theme", "sphinx (==5.0.2)", "sphinxcontrib-spelling"]
+qtpdf = ["pyqtwebengine (>=5.15)"]
+qtpng = ["pyqtwebengine (>=5.15)"]
+serve = ["tornado (>=6.1)"]
+test = ["flaky", "ipykernel", "ipywidgets (>=7.5)", "pytest (>=7)"]
+webpdf = ["playwright"]
+
+[[package]]
+name = "nbformat"
+version = "5.10.4"
+description = "The Jupyter Notebook format"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "nbformat-5.10.4-py3-none-any.whl", hash = "sha256:3b48d6c8fbca4b299bf3982ea7db1af21580e4fec269ad087b9e81588891200b"},
+ {file = "nbformat-5.10.4.tar.gz", hash = "sha256:322168b14f937a5d11362988ecac2a4952d3d8e3a2cbeb2319584631226d5b3a"},
+]
+
+[package.dependencies]
+fastjsonschema = ">=2.15"
+jsonschema = ">=2.6"
+jupyter-core = ">=4.12,<5.0.dev0 || >=5.1.dev0"
+traitlets = ">=5.1"
+
+[package.extras]
+docs = ["myst-parser", "pydata-sphinx-theme", "sphinx", "sphinxcontrib-github-alt", "sphinxcontrib-spelling"]
+test = ["pep440", "pre-commit", "pytest", "testpath"]
+
[[package]]
name = "packaging"
version = "24.1"
@@ -490,6 +975,17 @@ files = [
{file = "packaging-24.1.tar.gz", hash = "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002"},
]
+[[package]]
+name = "pandocfilters"
+version = "1.5.1"
+description = "Utilities for writing pandoc filters in python"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+files = [
+ {file = "pandocfilters-1.5.1-py2.py3-none-any.whl", hash = "sha256:93be382804a9cdb0a7267585f157e5d1731bbe5545a85b268d6f5fe6232de2bc"},
+ {file = "pandocfilters-1.5.1.tar.gz", hash = "sha256:002b4a555ee4ebc03f8b66307e287fa492e4a77b4ea14d3f934328297bb4939e"},
+]
+
[[package]]
name = "parameterized"
version = "0.9.0"
@@ -531,6 +1027,17 @@ docs = ["furo (>=2024.8.6)", "proselint (>=0.14)", "sphinx (>=8.0.2)", "sphinx-a
test = ["appdirs (==1.4.4)", "covdefaults (>=2.3)", "pytest (>=8.3.2)", "pytest-cov (>=5)", "pytest-mock (>=3.14)"]
type = ["mypy (>=1.11.2)"]
+[[package]]
+name = "pycparser"
+version = "2.22"
+description = "C parser in Python"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc"},
+ {file = "pycparser-2.22.tar.gz", hash = "sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6"},
+]
+
[[package]]
name = "pydantic"
version = "2.9.2"
@@ -655,6 +1162,79 @@ files = [
[package.dependencies]
typing-extensions = ">=4.6.0,<4.7.0 || >4.7.0"
+[[package]]
+name = "pygments"
+version = "2.18.0"
+description = "Pygments is a syntax highlighting package written in Python."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "pygments-2.18.0-py3-none-any.whl", hash = "sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a"},
+ {file = "pygments-2.18.0.tar.gz", hash = "sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199"},
+]
+
+[package.extras]
+windows-terminal = ["colorama (>=0.4.6)"]
+
+[[package]]
+name = "python-dateutil"
+version = "2.9.0.post0"
+description = "Extensions to the standard Python datetime module"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7"
+files = [
+ {file = "python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3"},
+ {file = "python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427"},
+]
+
+[package.dependencies]
+six = ">=1.5"
+
+[[package]]
+name = "python-frontmatter"
+version = "1.1.0"
+description = "Parse and manage posts with YAML (or other) frontmatter"
+optional = false
+python-versions = "*"
+files = [
+ {file = "python-frontmatter-1.1.0.tar.gz", hash = "sha256:7118d2bd56af9149625745c58c9b51fb67e8d1294a0c76796dafdc72c36e5f6d"},
+ {file = "python_frontmatter-1.1.0-py3-none-any.whl", hash = "sha256:335465556358d9d0e6c98bbeb69b1c969f2a4a21360587b9873bfc3b213407c1"},
+]
+
+[package.dependencies]
+PyYAML = "*"
+
+[package.extras]
+docs = ["sphinx"]
+test = ["mypy", "pyaml", "pytest", "toml", "types-PyYAML", "types-toml"]
+
+[[package]]
+name = "pywin32"
+version = "308"
+description = "Python for Window Extensions"
+optional = false
+python-versions = "*"
+files = [
+ {file = "pywin32-308-cp310-cp310-win32.whl", hash = "sha256:796ff4426437896550d2981b9c2ac0ffd75238ad9ea2d3bfa67a1abd546d262e"},
+ {file = "pywin32-308-cp310-cp310-win_amd64.whl", hash = "sha256:4fc888c59b3c0bef905ce7eb7e2106a07712015ea1c8234b703a088d46110e8e"},
+ {file = "pywin32-308-cp310-cp310-win_arm64.whl", hash = "sha256:a5ab5381813b40f264fa3495b98af850098f814a25a63589a8e9eb12560f450c"},
+ {file = "pywin32-308-cp311-cp311-win32.whl", hash = "sha256:5d8c8015b24a7d6855b1550d8e660d8daa09983c80e5daf89a273e5c6fb5095a"},
+ {file = "pywin32-308-cp311-cp311-win_amd64.whl", hash = "sha256:575621b90f0dc2695fec346b2d6302faebd4f0f45c05ea29404cefe35d89442b"},
+ {file = "pywin32-308-cp311-cp311-win_arm64.whl", hash = "sha256:100a5442b7332070983c4cd03f2e906a5648a5104b8a7f50175f7906efd16bb6"},
+ {file = "pywin32-308-cp312-cp312-win32.whl", hash = "sha256:587f3e19696f4bf96fde9d8a57cec74a57021ad5f204c9e627e15c33ff568897"},
+ {file = "pywin32-308-cp312-cp312-win_amd64.whl", hash = "sha256:00b3e11ef09ede56c6a43c71f2d31857cf7c54b0ab6e78ac659497abd2834f47"},
+ {file = "pywin32-308-cp312-cp312-win_arm64.whl", hash = "sha256:9b4de86c8d909aed15b7011182c8cab38c8850de36e6afb1f0db22b8959e3091"},
+ {file = "pywin32-308-cp313-cp313-win32.whl", hash = "sha256:1c44539a37a5b7b21d02ab34e6a4d314e0788f1690d65b48e9b0b89f31abbbed"},
+ {file = "pywin32-308-cp313-cp313-win_amd64.whl", hash = "sha256:fd380990e792eaf6827fcb7e187b2b4b1cede0585e3d0c9e84201ec27b9905e4"},
+ {file = "pywin32-308-cp313-cp313-win_arm64.whl", hash = "sha256:ef313c46d4c18dfb82a2431e3051ac8f112ccee1a34f29c263c583c568db63cd"},
+ {file = "pywin32-308-cp37-cp37m-win32.whl", hash = "sha256:1f696ab352a2ddd63bd07430080dd598e6369152ea13a25ebcdd2f503a38f1ff"},
+ {file = "pywin32-308-cp37-cp37m-win_amd64.whl", hash = "sha256:13dcb914ed4347019fbec6697a01a0aec61019c1046c2b905410d197856326a6"},
+ {file = "pywin32-308-cp38-cp38-win32.whl", hash = "sha256:5794e764ebcabf4ff08c555b31bd348c9025929371763b2183172ff4708152f0"},
+ {file = "pywin32-308-cp38-cp38-win_amd64.whl", hash = "sha256:3b92622e29d651c6b783e368ba7d6722b1634b8e70bd376fd7610fe1992e19de"},
+ {file = "pywin32-308-cp39-cp39-win32.whl", hash = "sha256:7873ca4dc60ab3287919881a7d4f88baee4a6e639aa6962de25a98ba6b193341"},
+ {file = "pywin32-308-cp39-cp39-win_amd64.whl", hash = "sha256:71b3322d949b4cc20776436a9c9ba0eeedcbc9c650daa536df63f0ff111bb920"},
+]
+
[[package]]
name = "pyyaml"
version = "6.0.2"
@@ -717,6 +1297,142 @@ files = [
{file = "pyyaml-6.0.2.tar.gz", hash = "sha256:d584d9ec91ad65861cc08d42e834324ef890a082e591037abe114850ff7bbc3e"},
]
+[[package]]
+name = "pyzmq"
+version = "26.2.0"
+description = "Python bindings for 0MQ"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "pyzmq-26.2.0-cp310-cp310-macosx_10_15_universal2.whl", hash = "sha256:ddf33d97d2f52d89f6e6e7ae66ee35a4d9ca6f36eda89c24591b0c40205a3629"},
+ {file = "pyzmq-26.2.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:dacd995031a01d16eec825bf30802fceb2c3791ef24bcce48fa98ce40918c27b"},
+ {file = "pyzmq-26.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:89289a5ee32ef6c439086184529ae060c741334b8970a6855ec0b6ad3ff28764"},
+ {file = "pyzmq-26.2.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5506f06d7dc6ecf1efacb4a013b1f05071bb24b76350832c96449f4a2d95091c"},
+ {file = "pyzmq-26.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8ea039387c10202ce304af74def5021e9adc6297067f3441d348d2b633e8166a"},
+ {file = "pyzmq-26.2.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:a2224fa4a4c2ee872886ed00a571f5e967c85e078e8e8c2530a2fb01b3309b88"},
+ {file = "pyzmq-26.2.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:28ad5233e9c3b52d76196c696e362508959741e1a005fb8fa03b51aea156088f"},
+ {file = "pyzmq-26.2.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:1c17211bc037c7d88e85ed8b7d8f7e52db6dc8eca5590d162717c654550f7282"},
+ {file = "pyzmq-26.2.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:b8f86dd868d41bea9a5f873ee13bf5551c94cf6bc51baebc6f85075971fe6eea"},
+ {file = "pyzmq-26.2.0-cp310-cp310-win32.whl", hash = "sha256:46a446c212e58456b23af260f3d9fb785054f3e3653dbf7279d8f2b5546b21c2"},
+ {file = "pyzmq-26.2.0-cp310-cp310-win_amd64.whl", hash = "sha256:49d34ab71db5a9c292a7644ce74190b1dd5a3475612eefb1f8be1d6961441971"},
+ {file = "pyzmq-26.2.0-cp310-cp310-win_arm64.whl", hash = "sha256:bfa832bfa540e5b5c27dcf5de5d82ebc431b82c453a43d141afb1e5d2de025fa"},
+ {file = "pyzmq-26.2.0-cp311-cp311-macosx_10_15_universal2.whl", hash = "sha256:8f7e66c7113c684c2b3f1c83cdd3376103ee0ce4c49ff80a648643e57fb22218"},
+ {file = "pyzmq-26.2.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:3a495b30fc91db2db25120df5847d9833af237546fd59170701acd816ccc01c4"},
+ {file = "pyzmq-26.2.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77eb0968da535cba0470a5165468b2cac7772cfb569977cff92e240f57e31bef"},
+ {file = "pyzmq-26.2.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6ace4f71f1900a548f48407fc9be59c6ba9d9aaf658c2eea6cf2779e72f9f317"},
+ {file = "pyzmq-26.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:92a78853d7280bffb93df0a4a6a2498cba10ee793cc8076ef797ef2f74d107cf"},
+ {file = "pyzmq-26.2.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:689c5d781014956a4a6de61d74ba97b23547e431e9e7d64f27d4922ba96e9d6e"},
+ {file = "pyzmq-26.2.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:0aca98bc423eb7d153214b2df397c6421ba6373d3397b26c057af3c904452e37"},
+ {file = "pyzmq-26.2.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:1f3496d76b89d9429a656293744ceca4d2ac2a10ae59b84c1da9b5165f429ad3"},
+ {file = "pyzmq-26.2.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:5c2b3bfd4b9689919db068ac6c9911f3fcb231c39f7dd30e3138be94896d18e6"},
+ {file = "pyzmq-26.2.0-cp311-cp311-win32.whl", hash = "sha256:eac5174677da084abf378739dbf4ad245661635f1600edd1221f150b165343f4"},
+ {file = "pyzmq-26.2.0-cp311-cp311-win_amd64.whl", hash = "sha256:5a509df7d0a83a4b178d0f937ef14286659225ef4e8812e05580776c70e155d5"},
+ {file = "pyzmq-26.2.0-cp311-cp311-win_arm64.whl", hash = "sha256:c0e6091b157d48cbe37bd67233318dbb53e1e6327d6fc3bb284afd585d141003"},
+ {file = "pyzmq-26.2.0-cp312-cp312-macosx_10_15_universal2.whl", hash = "sha256:ded0fc7d90fe93ae0b18059930086c51e640cdd3baebdc783a695c77f123dcd9"},
+ {file = "pyzmq-26.2.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:17bf5a931c7f6618023cdacc7081f3f266aecb68ca692adac015c383a134ca52"},
+ {file = "pyzmq-26.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55cf66647e49d4621a7e20c8d13511ef1fe1efbbccf670811864452487007e08"},
+ {file = "pyzmq-26.2.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4661c88db4a9e0f958c8abc2b97472e23061f0bc737f6f6179d7a27024e1faa5"},
+ {file = "pyzmq-26.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ea7f69de383cb47522c9c208aec6dd17697db7875a4674c4af3f8cfdac0bdeae"},
+ {file = "pyzmq-26.2.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:7f98f6dfa8b8ccaf39163ce872bddacca38f6a67289116c8937a02e30bbe9711"},
+ {file = "pyzmq-26.2.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:e3e0210287329272539eea617830a6a28161fbbd8a3271bf4150ae3e58c5d0e6"},
+ {file = "pyzmq-26.2.0-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:6b274e0762c33c7471f1a7471d1a2085b1a35eba5cdc48d2ae319f28b6fc4de3"},
+ {file = "pyzmq-26.2.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:29c6a4635eef69d68a00321e12a7d2559fe2dfccfa8efae3ffb8e91cd0b36a8b"},
+ {file = "pyzmq-26.2.0-cp312-cp312-win32.whl", hash = "sha256:989d842dc06dc59feea09e58c74ca3e1678c812a4a8a2a419046d711031f69c7"},
+ {file = "pyzmq-26.2.0-cp312-cp312-win_amd64.whl", hash = "sha256:2a50625acdc7801bc6f74698c5c583a491c61d73c6b7ea4dee3901bb99adb27a"},
+ {file = "pyzmq-26.2.0-cp312-cp312-win_arm64.whl", hash = "sha256:4d29ab8592b6ad12ebbf92ac2ed2bedcfd1cec192d8e559e2e099f648570e19b"},
+ {file = "pyzmq-26.2.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:9dd8cd1aeb00775f527ec60022004d030ddc51d783d056e3e23e74e623e33726"},
+ {file = "pyzmq-26.2.0-cp313-cp313-macosx_10_15_universal2.whl", hash = "sha256:28c812d9757fe8acecc910c9ac9dafd2ce968c00f9e619db09e9f8f54c3a68a3"},
+ {file = "pyzmq-26.2.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4d80b1dd99c1942f74ed608ddb38b181b87476c6a966a88a950c7dee118fdf50"},
+ {file = "pyzmq-26.2.0-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8c997098cc65e3208eca09303630e84d42718620e83b733d0fd69543a9cab9cb"},
+ {file = "pyzmq-26.2.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7ad1bc8d1b7a18497dda9600b12dc193c577beb391beae5cd2349184db40f187"},
+ {file = "pyzmq-26.2.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:bea2acdd8ea4275e1278350ced63da0b166421928276c7c8e3f9729d7402a57b"},
+ {file = "pyzmq-26.2.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:23f4aad749d13698f3f7b64aad34f5fc02d6f20f05999eebc96b89b01262fb18"},
+ {file = "pyzmq-26.2.0-cp313-cp313-musllinux_1_1_i686.whl", hash = "sha256:a4f96f0d88accc3dbe4a9025f785ba830f968e21e3e2c6321ccdfc9aef755115"},
+ {file = "pyzmq-26.2.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:ced65e5a985398827cc9276b93ef6dfabe0273c23de8c7931339d7e141c2818e"},
+ {file = "pyzmq-26.2.0-cp313-cp313-win32.whl", hash = "sha256:31507f7b47cc1ead1f6e86927f8ebb196a0bab043f6345ce070f412a59bf87b5"},
+ {file = "pyzmq-26.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:70fc7fcf0410d16ebdda9b26cbd8bf8d803d220a7f3522e060a69a9c87bf7bad"},
+ {file = "pyzmq-26.2.0-cp313-cp313-win_arm64.whl", hash = "sha256:c3789bd5768ab5618ebf09cef6ec2b35fed88709b104351748a63045f0ff9797"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:034da5fc55d9f8da09015d368f519478a52675e558c989bfcb5cf6d4e16a7d2a"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-macosx_10_15_universal2.whl", hash = "sha256:c92d73464b886931308ccc45b2744e5968cbaade0b1d6aeb40d8ab537765f5bc"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:794a4562dcb374f7dbbfb3f51d28fb40123b5a2abadee7b4091f93054909add5"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:aee22939bb6075e7afededabad1a56a905da0b3c4e3e0c45e75810ebe3a52672"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2ae90ff9dad33a1cfe947d2c40cb9cb5e600d759ac4f0fd22616ce6540f72797"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:43a47408ac52647dfabbc66a25b05b6a61700b5165807e3fbd40063fcaf46386"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:25bf2374a2a8433633c65ccb9553350d5e17e60c8eb4de4d92cc6bd60f01d306"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-musllinux_1_1_i686.whl", hash = "sha256:007137c9ac9ad5ea21e6ad97d3489af654381324d5d3ba614c323f60dab8fae6"},
+ {file = "pyzmq-26.2.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:470d4a4f6d48fb34e92d768b4e8a5cc3780db0d69107abf1cd7ff734b9766eb0"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:3b55a4229ce5da9497dd0452b914556ae58e96a4381bb6f59f1305dfd7e53fc8"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:9cb3a6460cdea8fe8194a76de8895707e61ded10ad0be97188cc8463ffa7e3a8"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8ab5cad923cc95c87bffee098a27856c859bd5d0af31bd346035aa816b081fe1"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9ed69074a610fad1c2fda66180e7b2edd4d31c53f2d1872bc2d1211563904cd9"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:cccba051221b916a4f5e538997c45d7d136a5646442b1231b916d0164067ea27"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:0eaa83fc4c1e271c24eaf8fb083cbccef8fde77ec8cd45f3c35a9a123e6da097"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:9edda2df81daa129b25a39b86cb57dfdfe16f7ec15b42b19bfac503360d27a93"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-win32.whl", hash = "sha256:ea0eb6af8a17fa272f7b98d7bebfab7836a0d62738e16ba380f440fceca2d951"},
+ {file = "pyzmq-26.2.0-cp37-cp37m-win_amd64.whl", hash = "sha256:4ff9dc6bc1664bb9eec25cd17506ef6672d506115095411e237d571e92a58231"},
+ {file = "pyzmq-26.2.0-cp38-cp38-macosx_10_15_universal2.whl", hash = "sha256:2eb7735ee73ca1b0d71e0e67c3739c689067f055c764f73aac4cc8ecf958ee3f"},
+ {file = "pyzmq-26.2.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:1a534f43bc738181aa7cbbaf48e3eca62c76453a40a746ab95d4b27b1111a7d2"},
+ {file = "pyzmq-26.2.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:aedd5dd8692635813368e558a05266b995d3d020b23e49581ddd5bbe197a8ab6"},
+ {file = "pyzmq-26.2.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8be4700cd8bb02cc454f630dcdf7cfa99de96788b80c51b60fe2fe1dac480289"},
+ {file = "pyzmq-26.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1fcc03fa4997c447dce58264e93b5aa2d57714fbe0f06c07b7785ae131512732"},
+ {file = "pyzmq-26.2.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:402b190912935d3db15b03e8f7485812db350d271b284ded2b80d2e5704be780"},
+ {file = "pyzmq-26.2.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:8685fa9c25ff00f550c1fec650430c4b71e4e48e8d852f7ddcf2e48308038640"},
+ {file = "pyzmq-26.2.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:76589c020680778f06b7e0b193f4b6dd66d470234a16e1df90329f5e14a171cd"},
+ {file = "pyzmq-26.2.0-cp38-cp38-win32.whl", hash = "sha256:8423c1877d72c041f2c263b1ec6e34360448decfb323fa8b94e85883043ef988"},
+ {file = "pyzmq-26.2.0-cp38-cp38-win_amd64.whl", hash = "sha256:76589f2cd6b77b5bdea4fca5992dc1c23389d68b18ccc26a53680ba2dc80ff2f"},
+ {file = "pyzmq-26.2.0-cp39-cp39-macosx_10_15_universal2.whl", hash = "sha256:b1d464cb8d72bfc1a3adc53305a63a8e0cac6bc8c5a07e8ca190ab8d3faa43c2"},
+ {file = "pyzmq-26.2.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:4da04c48873a6abdd71811c5e163bd656ee1b957971db7f35140a2d573f6949c"},
+ {file = "pyzmq-26.2.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:d049df610ac811dcffdc147153b414147428567fbbc8be43bb8885f04db39d98"},
+ {file = "pyzmq-26.2.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:05590cdbc6b902101d0e65d6a4780af14dc22914cc6ab995d99b85af45362cc9"},
+ {file = "pyzmq-26.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c811cfcd6a9bf680236c40c6f617187515269ab2912f3d7e8c0174898e2519db"},
+ {file = "pyzmq-26.2.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:6835dd60355593de10350394242b5757fbbd88b25287314316f266e24c61d073"},
+ {file = "pyzmq-26.2.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:bc6bee759a6bddea5db78d7dcd609397449cb2d2d6587f48f3ca613b19410cfc"},
+ {file = "pyzmq-26.2.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:c530e1eecd036ecc83c3407f77bb86feb79916d4a33d11394b8234f3bd35b940"},
+ {file = "pyzmq-26.2.0-cp39-cp39-win32.whl", hash = "sha256:367b4f689786fca726ef7a6c5ba606958b145b9340a5e4808132cc65759abd44"},
+ {file = "pyzmq-26.2.0-cp39-cp39-win_amd64.whl", hash = "sha256:e6fa2e3e683f34aea77de8112f6483803c96a44fd726d7358b9888ae5bb394ec"},
+ {file = "pyzmq-26.2.0-cp39-cp39-win_arm64.whl", hash = "sha256:7445be39143a8aa4faec43b076e06944b8f9d0701b669df4af200531b21e40bb"},
+ {file = "pyzmq-26.2.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:706e794564bec25819d21a41c31d4df2d48e1cc4b061e8d345d7fb4dd3e94072"},
+ {file = "pyzmq-26.2.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b435f2753621cd36e7c1762156815e21c985c72b19135dac43a7f4f31d28dd1"},
+ {file = "pyzmq-26.2.0-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:160c7e0a5eb178011e72892f99f918c04a131f36056d10d9c1afb223fc952c2d"},
+ {file = "pyzmq-26.2.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2c4a71d5d6e7b28a47a394c0471b7e77a0661e2d651e7ae91e0cab0a587859ca"},
+ {file = "pyzmq-26.2.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:90412f2db8c02a3864cbfc67db0e3dcdbda336acf1c469526d3e869394fe001c"},
+ {file = "pyzmq-26.2.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:2ea4ad4e6a12e454de05f2949d4beddb52460f3de7c8b9d5c46fbb7d7222e02c"},
+ {file = "pyzmq-26.2.0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:fc4f7a173a5609631bb0c42c23d12c49df3966f89f496a51d3eb0ec81f4519d6"},
+ {file = "pyzmq-26.2.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:878206a45202247781472a2d99df12a176fef806ca175799e1c6ad263510d57c"},
+ {file = "pyzmq-26.2.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:17c412bad2eb9468e876f556eb4ee910e62d721d2c7a53c7fa31e643d35352e6"},
+ {file = "pyzmq-26.2.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:0d987a3ae5a71c6226b203cfd298720e0086c7fe7c74f35fa8edddfbd6597eed"},
+ {file = "pyzmq-26.2.0-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:39887ac397ff35b7b775db7201095fc6310a35fdbae85bac4523f7eb3b840e20"},
+ {file = "pyzmq-26.2.0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:fdb5b3e311d4d4b0eb8b3e8b4d1b0a512713ad7e6a68791d0923d1aec433d919"},
+ {file = "pyzmq-26.2.0-pp38-pypy38_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:226af7dcb51fdb0109f0016449b357e182ea0ceb6b47dfb5999d569e5db161d5"},
+ {file = "pyzmq-26.2.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0bed0e799e6120b9c32756203fb9dfe8ca2fb8467fed830c34c877e25638c3fc"},
+ {file = "pyzmq-26.2.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:29c7947c594e105cb9e6c466bace8532dc1ca02d498684128b339799f5248277"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-macosx_10_15_x86_64.whl", hash = "sha256:cdeabcff45d1c219636ee2e54d852262e5c2e085d6cb476d938aee8d921356b3"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:35cffef589bcdc587d06f9149f8d5e9e8859920a071df5a2671de2213bef592a"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:18c8dc3b7468d8b4bdf60ce9d7141897da103c7a4690157b32b60acb45e333e6"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7133d0a1677aec369d67dd78520d3fa96dd7f3dcec99d66c1762870e5ea1a50a"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:6a96179a24b14fa6428cbfc08641c779a53f8fcec43644030328f44034c7f1f4"},
+ {file = "pyzmq-26.2.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:4f78c88905461a9203eac9faac157a2a0dbba84a0fd09fd29315db27be40af9f"},
+ {file = "pyzmq-26.2.0.tar.gz", hash = "sha256:070672c258581c8e4f640b5159297580a9974b026043bd4ab0470be9ed324f1f"},
+]
+
+[package.dependencies]
+cffi = {version = "*", markers = "implementation_name == \"pypy\""}
+
+[[package]]
+name = "referencing"
+version = "0.35.1"
+description = "JSON Referencing + Python"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "referencing-0.35.1-py3-none-any.whl", hash = "sha256:eda6d3234d62814d1c64e305c1331c9a3a6132da475ab6382eaa997b21ee75de"},
+ {file = "referencing-0.35.1.tar.gz", hash = "sha256:25b42124a6c8b632a425174f24087783efb348a6f1e0008e63cd4466fedf703c"},
+]
+
+[package.dependencies]
+attrs = ">=22.2.0"
+rpds-py = ">=0.7.0"
+
[[package]]
name = "requests"
version = "2.32.3"
@@ -738,6 +1454,129 @@ urllib3 = ">=1.21.1,<3"
socks = ["PySocks (>=1.5.6,!=1.5.7)"]
use-chardet-on-py3 = ["chardet (>=3.0.2,<6)"]
+[[package]]
+name = "rpds-py"
+version = "0.22.3"
+description = "Python bindings to Rust's persistent data structures (rpds)"
+optional = false
+python-versions = ">=3.9"
+files = [
+ {file = "rpds_py-0.22.3-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:6c7b99ca52c2c1752b544e310101b98a659b720b21db00e65edca34483259967"},
+ {file = "rpds_py-0.22.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:be2eb3f2495ba669d2a985f9b426c1797b7d48d6963899276d22f23e33d47e37"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:70eb60b3ae9245ddea20f8a4190bd79c705a22f8028aaf8bbdebe4716c3fab24"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4041711832360a9b75cfb11b25a6a97c8fb49c07b8bd43d0d02b45d0b499a4ff"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:64607d4cbf1b7e3c3c8a14948b99345eda0e161b852e122c6bb71aab6d1d798c"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:81e69b0a0e2537f26d73b4e43ad7bc8c8efb39621639b4434b76a3de50c6966e"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc27863442d388870c1809a87507727b799c8460573cfbb6dc0eeaef5a11b5ec"},
+ {file = "rpds_py-0.22.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e79dd39f1e8c3504be0607e5fc6e86bb60fe3584bec8b782578c3b0fde8d932c"},
+ {file = "rpds_py-0.22.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e0fa2d4ec53dc51cf7d3bb22e0aa0143966119f42a0c3e4998293a3dd2856b09"},
+ {file = "rpds_py-0.22.3-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:fda7cb070f442bf80b642cd56483b5548e43d366fe3f39b98e67cce780cded00"},
+ {file = "rpds_py-0.22.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:cff63a0272fcd259dcc3be1657b07c929c466b067ceb1c20060e8d10af56f5bf"},
+ {file = "rpds_py-0.22.3-cp310-cp310-win32.whl", hash = "sha256:9bd7228827ec7bb817089e2eb301d907c0d9827a9e558f22f762bb690b131652"},
+ {file = "rpds_py-0.22.3-cp310-cp310-win_amd64.whl", hash = "sha256:9beeb01d8c190d7581a4d59522cd3d4b6887040dcfc744af99aa59fef3e041a8"},
+ {file = "rpds_py-0.22.3-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:d20cfb4e099748ea39e6f7b16c91ab057989712d31761d3300d43134e26e165f"},
+ {file = "rpds_py-0.22.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:68049202f67380ff9aa52f12e92b1c30115f32e6895cd7198fa2a7961621fc5a"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fb4f868f712b2dd4bcc538b0a0c1f63a2b1d584c925e69a224d759e7070a12d5"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bc51abd01f08117283c5ebf64844a35144a0843ff7b2983e0648e4d3d9f10dbb"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0f3cec041684de9a4684b1572fe28c7267410e02450f4561700ca5a3bc6695a2"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7ef9d9da710be50ff6809fed8f1963fecdfecc8b86656cadfca3bc24289414b0"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:59f4a79c19232a5774aee369a0c296712ad0e77f24e62cad53160312b1c1eaa1"},
+ {file = "rpds_py-0.22.3-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1a60bce91f81ddaac922a40bbb571a12c1070cb20ebd6d49c48e0b101d87300d"},
+ {file = "rpds_py-0.22.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e89391e6d60251560f0a8f4bd32137b077a80d9b7dbe6d5cab1cd80d2746f648"},
+ {file = "rpds_py-0.22.3-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:e3fb866d9932a3d7d0c82da76d816996d1667c44891bd861a0f97ba27e84fc74"},
+ {file = "rpds_py-0.22.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1352ae4f7c717ae8cba93421a63373e582d19d55d2ee2cbb184344c82d2ae55a"},
+ {file = "rpds_py-0.22.3-cp311-cp311-win32.whl", hash = "sha256:b0b4136a252cadfa1adb705bb81524eee47d9f6aab4f2ee4fa1e9d3cd4581f64"},
+ {file = "rpds_py-0.22.3-cp311-cp311-win_amd64.whl", hash = "sha256:8bd7c8cfc0b8247c8799080fbff54e0b9619e17cdfeb0478ba7295d43f635d7c"},
+ {file = "rpds_py-0.22.3-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:27e98004595899949bd7a7b34e91fa7c44d7a97c40fcaf1d874168bb652ec67e"},
+ {file = "rpds_py-0.22.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1978d0021e943aae58b9b0b196fb4895a25cc53d3956b8e35e0b7682eefb6d56"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:655ca44a831ecb238d124e0402d98f6212ac527a0ba6c55ca26f616604e60a45"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:feea821ee2a9273771bae61194004ee2fc33f8ec7db08117ef9147d4bbcbca8e"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:22bebe05a9ffc70ebfa127efbc429bc26ec9e9b4ee4d15a740033efda515cf3d"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3af6e48651c4e0d2d166dc1b033b7042ea3f871504b6805ba5f4fe31581d8d38"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e67ba3c290821343c192f7eae1d8fd5999ca2dc99994114643e2f2d3e6138b15"},
+ {file = "rpds_py-0.22.3-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:02fbb9c288ae08bcb34fb41d516d5eeb0455ac35b5512d03181d755d80810059"},
+ {file = "rpds_py-0.22.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f56a6b404f74ab372da986d240e2e002769a7d7102cc73eb238a4f72eec5284e"},
+ {file = "rpds_py-0.22.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:0a0461200769ab3b9ab7e513f6013b7a97fdeee41c29b9db343f3c5a8e2b9e61"},
+ {file = "rpds_py-0.22.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:8633e471c6207a039eff6aa116e35f69f3156b3989ea3e2d755f7bc41754a4a7"},
+ {file = "rpds_py-0.22.3-cp312-cp312-win32.whl", hash = "sha256:593eba61ba0c3baae5bc9be2f5232430453fb4432048de28399ca7376de9c627"},
+ {file = "rpds_py-0.22.3-cp312-cp312-win_amd64.whl", hash = "sha256:d115bffdd417c6d806ea9069237a4ae02f513b778e3789a359bc5856e0404cc4"},
+ {file = "rpds_py-0.22.3-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:ea7433ce7e4bfc3a85654aeb6747babe3f66eaf9a1d0c1e7a4435bbdf27fea84"},
+ {file = "rpds_py-0.22.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6dd9412824c4ce1aca56c47b0991e65bebb7ac3f4edccfd3f156150c96a7bf25"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:20070c65396f7373f5df4005862fa162db5d25d56150bddd0b3e8214e8ef45b4"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0b09865a9abc0ddff4e50b5ef65467cd94176bf1e0004184eb915cbc10fc05c5"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3453e8d41fe5f17d1f8e9c383a7473cd46a63661628ec58e07777c2fff7196dc"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f5d36399a1b96e1a5fdc91e0522544580dbebeb1f77f27b2b0ab25559e103b8b"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:009de23c9c9ee54bf11303a966edf4d9087cd43a6003672e6aa7def643d06518"},
+ {file = "rpds_py-0.22.3-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1aef18820ef3e4587ebe8b3bc9ba6e55892a6d7b93bac6d29d9f631a3b4befbd"},
+ {file = "rpds_py-0.22.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f60bd8423be1d9d833f230fdbccf8f57af322d96bcad6599e5a771b151398eb2"},
+ {file = "rpds_py-0.22.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:62d9cfcf4948683a18a9aff0ab7e1474d407b7bab2ca03116109f8464698ab16"},
+ {file = "rpds_py-0.22.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9253fc214112405f0afa7db88739294295f0e08466987f1d70e29930262b4c8f"},
+ {file = "rpds_py-0.22.3-cp313-cp313-win32.whl", hash = "sha256:fb0ba113b4983beac1a2eb16faffd76cb41e176bf58c4afe3e14b9c681f702de"},
+ {file = "rpds_py-0.22.3-cp313-cp313-win_amd64.whl", hash = "sha256:c58e2339def52ef6b71b8f36d13c3688ea23fa093353f3a4fee2556e62086ec9"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:f82a116a1d03628a8ace4859556fb39fd1424c933341a08ea3ed6de1edb0283b"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3dfcbc95bd7992b16f3f7ba05af8a64ca694331bd24f9157b49dadeeb287493b"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:59259dc58e57b10e7e18ce02c311804c10c5a793e6568f8af4dead03264584d1"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5725dd9cc02068996d4438d397e255dcb1df776b7ceea3b9cb972bdb11260a83"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:99b37292234e61325e7a5bb9689e55e48c3f5f603af88b1642666277a81f1fbd"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:27b1d3b3915a99208fee9ab092b8184c420f2905b7d7feb4aeb5e4a9c509b8a1"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f612463ac081803f243ff13cccc648578e2279295048f2a8d5eb430af2bae6e3"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f73d3fef726b3243a811121de45193c0ca75f6407fe66f3f4e183c983573e130"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:3f21f0495edea7fdbaaa87e633a8689cd285f8f4af5c869f27bc8074638ad69c"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:1e9663daaf7a63ceccbbb8e3808fe90415b0757e2abddbfc2e06c857bf8c5e2b"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a76e42402542b1fae59798fab64432b2d015ab9d0c8c47ba7addddbaf7952333"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-win32.whl", hash = "sha256:69803198097467ee7282750acb507fba35ca22cc3b85f16cf45fb01cb9097730"},
+ {file = "rpds_py-0.22.3-cp313-cp313t-win_amd64.whl", hash = "sha256:f5cf2a0c2bdadf3791b5c205d55a37a54025c6e18a71c71f82bb536cf9a454bf"},
+ {file = "rpds_py-0.22.3-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:378753b4a4de2a7b34063d6f95ae81bfa7b15f2c1a04a9518e8644e81807ebea"},
+ {file = "rpds_py-0.22.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:3445e07bf2e8ecfeef6ef67ac83de670358abf2996916039b16a218e3d95e97e"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7b2513ba235829860b13faa931f3b6846548021846ac808455301c23a101689d"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:eaf16ae9ae519a0e237a0f528fd9f0197b9bb70f40263ee57ae53c2b8d48aeb3"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:583f6a1993ca3369e0f80ba99d796d8e6b1a3a2a442dd4e1a79e652116413091"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4617e1915a539a0d9a9567795023de41a87106522ff83fbfaf1f6baf8e85437e"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c150c7a61ed4a4f4955a96626574e9baf1adf772c2fb61ef6a5027e52803543"},
+ {file = "rpds_py-0.22.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2fa4331c200c2521512595253f5bb70858b90f750d39b8cbfd67465f8d1b596d"},
+ {file = "rpds_py-0.22.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:214b7a953d73b5e87f0ebece4a32a5bd83c60a3ecc9d4ec8f1dca968a2d91e99"},
+ {file = "rpds_py-0.22.3-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:f47ad3d5f3258bd7058d2d506852217865afefe6153a36eb4b6928758041d831"},
+ {file = "rpds_py-0.22.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:f276b245347e6e36526cbd4a266a417796fc531ddf391e43574cf6466c492520"},
+ {file = "rpds_py-0.22.3-cp39-cp39-win32.whl", hash = "sha256:bbb232860e3d03d544bc03ac57855cd82ddf19c7a07651a7c0fdb95e9efea8b9"},
+ {file = "rpds_py-0.22.3-cp39-cp39-win_amd64.whl", hash = "sha256:cfbc454a2880389dbb9b5b398e50d439e2e58669160f27b60e5eca11f68ae17c"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:d48424e39c2611ee1b84ad0f44fb3b2b53d473e65de061e3f460fc0be5f1939d"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:24e8abb5878e250f2eb0d7859a8e561846f98910326d06c0d51381fed59357bd"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4b232061ca880db21fa14defe219840ad9b74b6158adb52ddf0e87bead9e8493"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac0a03221cdb5058ce0167ecc92a8c89e8d0decdc9e99a2ec23380793c4dcb96"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:eb0c341fa71df5a4595f9501df4ac5abfb5a09580081dffbd1ddd4654e6e9123"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bf9db5488121b596dbfc6718c76092fda77b703c1f7533a226a5a9f65248f8ad"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0b8db6b5b2d4491ad5b6bdc2bc7c017eec108acbf4e6785f42a9eb0ba234f4c9"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b3d504047aba448d70cf6fa22e06cb09f7cbd761939fdd47604f5e007675c24e"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:e61b02c3f7a1e0b75e20c3978f7135fd13cb6cf551bf4a6d29b999a88830a338"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-musllinux_1_2_i686.whl", hash = "sha256:e35ba67d65d49080e8e5a1dd40101fccdd9798adb9b050ff670b7d74fa41c566"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:26fd7cac7dd51011a245f29a2cc6489c4608b5a8ce8d75661bb4a1066c52dfbe"},
+ {file = "rpds_py-0.22.3-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:177c7c0fce2855833819c98e43c262007f42ce86651ffbb84f37883308cb0e7d"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:bb47271f60660803ad11f4c61b42242b8c1312a31c98c578f79ef9387bbde21c"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:70fb28128acbfd264eda9bf47015537ba3fe86e40d046eb2963d75024be4d055"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:44d61b4b7d0c2c9ac019c314e52d7cbda0ae31078aabd0f22e583af3e0d79723"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5f0e260eaf54380380ac3808aa4ebe2d8ca28b9087cf411649f96bad6900c728"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b25bc607423935079e05619d7de556c91fb6adeae9d5f80868dde3468657994b"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:fb6116dfb8d1925cbdb52595560584db42a7f664617a1f7d7f6e32f138cdf37d"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a63cbdd98acef6570c62b92a1e43266f9e8b21e699c363c0fef13bd530799c11"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2b8f60e1b739a74bab7e01fcbe3dddd4657ec685caa04681df9d562ef15b625f"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:2e8b55d8517a2fda8d95cb45d62a5a8bbf9dd0ad39c5b25c8833efea07b880ca"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-musllinux_1_2_i686.whl", hash = "sha256:2de29005e11637e7a2361fa151f780ff8eb2543a0da1413bb951e9f14b699ef3"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:666ecce376999bf619756a24ce15bb14c5bfaf04bf00abc7e663ce17c3f34fe7"},
+ {file = "rpds_py-0.22.3-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:5246b14ca64a8675e0a7161f7af68fe3e910e6b90542b4bfb5439ba752191df6"},
+ {file = "rpds_py-0.22.3.tar.gz", hash = "sha256:e32fee8ab45d3c2db6da19a5323bc3362237c8b653c70194414b892fd06a080d"},
+]
+
+[[package]]
+name = "six"
+version = "1.17.0"
+description = "Python 2 and 3 compatibility utilities"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7"
+files = [
+ {file = "six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274"},
+ {file = "six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81"},
+]
+
[[package]]
name = "sniffio"
version = "1.3.1"
@@ -749,6 +1588,35 @@ files = [
{file = "sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc"},
]
+[[package]]
+name = "soupsieve"
+version = "2.6"
+description = "A modern CSS selector implementation for Beautiful Soup."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "soupsieve-2.6-py3-none-any.whl", hash = "sha256:e72c4ff06e4fb6e4b5a9f0f55fe6e81514581fca1515028625d0f299c602ccc9"},
+ {file = "soupsieve-2.6.tar.gz", hash = "sha256:e2e68417777af359ec65daac1057404a3c8a5455bb8abc36f1a9866ab1a51abb"},
+]
+
+[[package]]
+name = "tinycss2"
+version = "1.4.0"
+description = "A tiny CSS parser"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "tinycss2-1.4.0-py3-none-any.whl", hash = "sha256:3a49cf47b7675da0b15d0c6e1df8df4ebd96e9394bb905a5775adb0d884c5289"},
+ {file = "tinycss2-1.4.0.tar.gz", hash = "sha256:10c0972f6fc0fbee87c3edb76549357415e94548c1ae10ebccdea16fb404a9b7"},
+]
+
+[package.dependencies]
+webencodings = ">=0.4"
+
+[package.extras]
+doc = ["sphinx", "sphinx_rtd_theme"]
+test = ["pytest", "ruff"]
+
[[package]]
name = "tokenizers"
version = "0.20.1"
@@ -866,6 +1734,26 @@ dev = ["tokenizers[testing]"]
docs = ["setuptools-rust", "sphinx", "sphinx-rtd-theme"]
testing = ["black (==22.3)", "datasets", "numpy", "pytest", "requests", "ruff"]
+[[package]]
+name = "tornado"
+version = "6.4.2"
+description = "Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed."
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "tornado-6.4.2-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:e828cce1123e9e44ae2a50a9de3055497ab1d0aeb440c5ac23064d9e44880da1"},
+ {file = "tornado-6.4.2-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:072ce12ada169c5b00b7d92a99ba089447ccc993ea2143c9ede887e0937aa803"},
+ {file = "tornado-6.4.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1a017d239bd1bb0919f72af256a970624241f070496635784d9bf0db640d3fec"},
+ {file = "tornado-6.4.2-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c36e62ce8f63409301537222faffcef7dfc5284f27eec227389f2ad11b09d946"},
+ {file = "tornado-6.4.2-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bca9eb02196e789c9cb5c3c7c0f04fb447dc2adffd95265b2c7223a8a615ccbf"},
+ {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:304463bd0772442ff4d0f5149c6f1c2135a1fae045adf070821c6cdc76980634"},
+ {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:c82c46813ba483a385ab2a99caeaedf92585a1f90defb5693351fa7e4ea0bf73"},
+ {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:932d195ca9015956fa502c6b56af9eb06106140d844a335590c1ec7f5277d10c"},
+ {file = "tornado-6.4.2-cp38-abi3-win32.whl", hash = "sha256:2876cef82e6c5978fde1e0d5b1f919d756968d5b4282418f3146b79b58556482"},
+ {file = "tornado-6.4.2-cp38-abi3-win_amd64.whl", hash = "sha256:908b71bf3ff37d81073356a5fadcc660eb10c1476ee6e2725588626ce7e5ca38"},
+ {file = "tornado-6.4.2.tar.gz", hash = "sha256:92bad5b4746e9879fd7bf1eb21dce4e3fc5128d71601f80005afa39237ad620b"},
+]
+
[[package]]
name = "tqdm"
version = "4.66.5"
@@ -886,6 +1774,21 @@ notebook = ["ipywidgets (>=6)"]
slack = ["slack-sdk"]
telegram = ["requests"]
+[[package]]
+name = "traitlets"
+version = "5.14.3"
+description = "Traitlets Python configuration system"
+optional = false
+python-versions = ">=3.8"
+files = [
+ {file = "traitlets-5.14.3-py3-none-any.whl", hash = "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f"},
+ {file = "traitlets-5.14.3.tar.gz", hash = "sha256:9ed0579d3502c94b4b3732ac120375cda96f923114522847de4b3bb98b96b6b7"},
+]
+
+[package.extras]
+docs = ["myst-parser", "pydata-sphinx-theme", "sphinx"]
+test = ["argcomplete (>=3.0.3)", "mypy (>=1.7.0)", "pre-commit", "pytest (>=7.0,<8.2)", "pytest-mock", "pytest-mypy-testing"]
+
[[package]]
name = "types-requests"
version = "2.32.0.20241016"
@@ -928,7 +1831,18 @@ h2 = ["h2 (>=4,<5)"]
socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"]
zstd = ["zstandard (>=0.18.0)"]
+[[package]]
+name = "webencodings"
+version = "0.5.1"
+description = "Character encoding aliases for legacy web content"
+optional = false
+python-versions = "*"
+files = [
+ {file = "webencodings-0.5.1-py2.py3-none-any.whl", hash = "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78"},
+ {file = "webencodings-0.5.1.tar.gz", hash = "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"},
+]
+
[metadata]
lock-version = "2.0"
python-versions = "^3.11"
-content-hash = "9f3e32193f943b04cabbcb26f2412fffadea02a184bdcee11907d419da4ce40e"
+content-hash = "6cd27e6040d352f255eb624544ad04e5577cf24a1139013a223cbd2c73b426ce"
diff --git a/pyproject.toml b/pyproject.toml
index 693342fed..b1225a999 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -13,6 +13,11 @@ package-mode = false
python = "^3.11"
cohere = "^5.11.1"
black = "^24.10.0"
+nbconvert = "^7.16.4"
+jinja2 = "^3.1.4"
+pyyaml = "^6.0.2"
+python-frontmatter = "^1.1.0"
+mypy = "^1.14.1"
[build-system]
diff --git a/scripts/build-cookbooks/authors.yaml b/scripts/build-cookbooks/authors.yaml
new file mode 100644
index 000000000..933458de5
--- /dev/null
+++ b/scripts/build-cookbooks/authors.yaml
@@ -0,0 +1,115 @@
+alexandrebarbet@cohere.com:
+ name: Alex Barbet
+ img: bf2c763-Alex.jpg
+
+neeral@cohere.com:
+ name: Neeral Beladia
+ img: a3689fe-Neeral.jpg
+
+harri@cohere.com:
+ name: Harri Bell-Thomas
+ img: 468e687-Harri.jpg
+
+anna@cohere.com:
+ name: Ania Bialas
+ img: c5dc5a3-Ania.jpg
+
+giannis@cohere.com:
+ name: Giannis Chatziveroglou
+ img: 73153cb-giannis.jpeg
+
+williamdarling@cohere.com:
+ name: William Darling
+ img: 7ee933d-William_Darling.jpg
+
+antoine@cohere.com:
+ name: Antoine Debugne
+ img: edbe0b1-Antoine.jpg
+
+marco@cohere.com:
+ name: Marco Del Tredici
+ img: f103c96-Marco.jpg
+
+shaan@cohere.com:
+ name: Shaan Desai
+ img: d17fc44-Shaan.jpg
+
+joan@cohere.com:
+ name: Joan Devassy
+ img: 648e4de-Kyle.jpg
+
+kyle@cohere.com:
+ name: Kyle Duffy
+ img: 648e4de-Kyle.jpg
+
+abel@cohere.com:
+ name: Abel Essiane
+ img: 7ad72f4-Abel.jpg
+
+boyu@cohere.com:
+ name: Boyu Fan
+ img: 66b2d90-boyufan.JPG.jpg
+
+utsav@cohere.com:
+ name: Utsav Garg
+ img: 9688517-Utsav.jpg
+
+jason@cohere.com:
+ name: Jason Jung
+ img: 0803e3d-Jason_Jung.jpg
+
+trushant@cohere.com:
+ name: Trushant Kalyanpur
+ img: de428e0-Trushant.jpg
+
+gokce@cohere.com:
+ name: Gokce Keskin
+ img: fe496a9-Gokce.jpg
+
+olivia@cohere.com:
+ name: Olivia Lasche
+ img: 7419032-Olivia.jpg
+
+justin@cohere.com:
+ name: Justin Lee
+ img: 3678fac-Justin.jpg
+
+katelubrano@cohere.com:
+ name: Kate Lubrano
+ img: 1131093-Kate.jpg
+
+lacey@cohere.com:
+ name: Lacey Mclear
+ img: 88c1977-Lacey.jpg
+
+vivek@cohere.com:
+ name: Vivek Mupalla
+ img: 3666c64-Vivek.jpg
+
+aal@cohere.com:
+ name: Aal Patankar
+ img: d48e622-Aal.jpg
+
+mauro@cohere.com:
+ name: Mauro Schilman
+ img: 16c3bde-Mauro.jpg
+
+sanal@cohere.com:
+ name: Sanal Shivaprasad
+ img: 811a185-Sanal.jpg
+
+anirudh@cohere.com:
+ name: Anirudh Shrinivason
+ img: e257c29-Anirudh.jpg
+
+vanessa@cohere.com:
+ name: Vanessa Tang
+ img: f6a8d8e-Vanessa.jpg
+
+jennifer@cohere.com:
+ name: Jennifer Tracey
+ img: 4579ada-Jennifer.jpg
+
+evren@cohere.com:
+ name: Evren Tumer
+ img: d69301f-Evren.jpg
diff --git a/scripts/build-cookbooks/build.py b/scripts/build-cookbooks/build.py
new file mode 100644
index 000000000..1263e1ee2
--- /dev/null
+++ b/scripts/build-cookbooks/build.py
@@ -0,0 +1,122 @@
+import yaml
+import nbformat
+from pathlib import Path
+from nbconvert import MarkdownExporter
+from jinja2 import Environment, FileSystemLoader
+import frontmatter
+import re
+from base64 import b64encode
+
+
+BASE_PATH = Path(__file__).resolve().parents[2]
+COOKBOOKS_PATH = BASE_PATH / "fern/pages/cookbooks"
+BUILD_COOKBOOKS_PATH = BASE_PATH / "scripts/build-cookbooks"
+REGISTRY_FILE = BUILD_COOKBOOKS_PATH / "registry.yaml"
+AUTHORS_FILE = BUILD_COOKBOOKS_PATH / "authors.yaml"
+TEMPLATES_PATH = BUILD_COOKBOOKS_PATH / "templates"
+MARKDOWN_IMAGE_IMPORT_PATTERN = re.compile(r"!\[(.*?)\]\((.*?)\)")
+TITLE_PATTERN = re.compile(r"(?m)^#\s.*\n", re.MULTILINE)
+SCRIPT_PATTERN = re.compile(r"", re.DOTALL)
+
+
+def _format_bytes_as_base64(data: bytes) -> str:
+ return b64encode(data).decode("utf-8")
+
+env = Environment(loader=FileSystemLoader(TEMPLATES_PATH))
+template = env.get_template('cookbook.md')
+
+def _post_process(body: str, resources: dict[str]) -> str:
+ """Perform any transformations to the generated body Markdown."""
+
+ body = re.sub(TITLE_PATTERN, "", body, count=1)
+ body = re.sub(SCRIPT_PATTERN, "", body)
+
+ # Convert the image data to base64 encoded strings
+ outputs = resources["outputs"]
+ pngs = [k for k in outputs.keys() if k.endswith(".png")]
+ png_src_dict = {
+ k: f"data:image/png;base64,{_format_bytes_as_base64(outputs[k])}" for k in pngs
+ }
+
+ # Replace markdown image links with img tags that include the appropriate base64 data
+ for k, v in png_src_dict.items():
+ body = body.replace(k, v)
+
+ # Format as img tags
+ def _sub(match):
+ alt, src = match.groups()
+ html = f'
'
+ return html
+
+ body = re.sub(MARKDOWN_IMAGE_IMPORT_PATTERN, _sub, body)
+
+ return body
+
+def build_cookbooks():
+ with open(REGISTRY_FILE, 'r') as file:
+ registry = yaml.safe_load(file)
+
+
+ # Open author metadata DB
+ with open(AUTHORS_FILE, "r") as f:
+ _authors: dict[dict] = yaml.safe_load(f)
+
+ md_exporter = MarkdownExporter()
+
+ for entry in registry:
+ if not entry.get("publish", True):
+ continue
+
+ author_email_list = entry.get("authors", list())
+ authors = [
+ dict(
+ name=_authors[email]["name"],
+ img=_authors[email]["img"],
+ email=email,
+ )
+ for email in author_email_list
+ if email in _authors
+ ]
+
+ notebook_path = BASE_PATH / entry["path"]
+ with open(notebook_path, 'r') as f:
+ notebook_content = nbformat.read(f, as_version=4)
+ body, resources = md_exporter.from_notebook_node(notebook_content)
+
+ # Post-process content
+ body = _post_process(body, resources)
+
+ slug = f"/page/{entry['slug']}"
+ output_file_path = COOKBOOKS_PATH / f"{entry['slug']}.mdx"
+
+ if output_file_path.exists():
+ with open(output_file_path, 'r', encoding='utf-8') as file:
+ post = frontmatter.load(file)
+ existing_metadata = post.metadata
+ else:
+ existing_metadata = {}
+
+ # Prepare data for rendering the template
+ body_data = {
+ 'title': entry.get('title', existing_metadata.get('title', 'Default Title')),
+ 'slug': slug,
+ 'description': entry.get('description', existing_metadata.get('description', '')),
+ 'image': entry.get('image', existing_metadata.get('image', '')),
+ 'keywords': entry.get('keywords', existing_metadata.get('keywords', '')),
+ 'body': body,
+ 'authors': authors,
+ 'cookbook_path': entry.get('path', '')
+ }
+
+ # Render the template with the data
+ content = template.render(body_data)
+
+ # Ensure the directory exists
+ output_file_path.parent.mkdir(parents=True, exist_ok=True)
+
+ # Write to file
+ with open(output_file_path, 'w', encoding='utf-8') as file:
+ file.write(content)
+
+if __name__ == "__main__":
+ build_cookbooks()
diff --git a/scripts/build-cookbooks/registry.yaml b/scripts/build-cookbooks/registry.yaml
new file mode 100644
index 000000000..015f4ef61
--- /dev/null
+++ b/scripts/build-cookbooks/registry.yaml
@@ -0,0 +1,437 @@
+## Agents
+- title: Calendar Agent with Native Multi Step Tool
+ path: notebooks/agents/Tool_Use.ipynb
+ slug: calendar-agent
+ blurb: A minimal working example of how to use our chat API to call tools.
+ tags:
+ - agents
+
+- title: Basic Tool Use
+ path: notebooks/agents/Vanilla_Tool_Use.ipynb
+ slug: basic-tool-use
+ blurb:
+ Connect large language models to external tools, like search engines, APIs,
+ and databases, to access and utilise a wider range of data.
+ tags:
+ - agents
+
+- title: Multi-Step Tool Use
+ path: notebooks/agents/Vanilla_Multi_Step_Tool_Use.ipynb
+ slug: basic-multi-step
+ blurb:
+ Multi-step tool use allows developers to connect Cohere's models to external
+ tools like search engines, APIs, and databases.
+ tags:
+ - agents
+ - oss
+
+- title: A ReAct Agent with Command R+, Achieving The Same Goals as Adaptive RAG
+ path: notebooks/react_agent_adaptive_rag_cohere.ipynb
+ slug: adaptive-rag-agent
+ blurb:
+ An example of using Command R+ to create an agent that can search the web,
+ retrieve info from a vector store, and self-reflect.
+ tags:
+ - agents
+ - rag
+ publish: false
+
+- title: A Data Analyst Agent Built with Cohere and Langchain
+ path: notebooks/agents/Data_Analyst_Agent_Cohere_and_Langchain.ipynb
+ slug: data-analyst-agent
+ blurb: Build a data analyst agent with Python and Cohere's Command R+ mode and Langchain.
+ tags:
+ - agents
+ - oss
+
+- title: Short-Term Memory Handling for Agents
+ path: notebooks/agents/agent_memory_walkthrough.ipynb
+ slug: agent-short-term-memory
+ blurb: A walkthrough of how to use Langchain cohere_react_agent to effectively manage short-term chat history that contains tool calls with Langchain.
+ tags:
+ - agents
+ authors:
+ - marco@cohere.com
+
+- title: Agent API Calls
+ path: notebooks/agents/agents_with_deterministic_functions.ipynb
+ slug: agent-api-calls
+ blurb: A walkthrough of how to use Langchain cohere_react_agent to make API calls to external services that require regex.
+ tags:
+ - agents
+ authors:
+ - marco@cohere.com
+
+- title: Financial CSV Agent with Langchain
+ path: notebooks/agents/financial-csv-agent/financial_csv_publication.ipynb
+ slug: csv-agent
+ blurb: The notebook demonstrates how to setup a Langchain Cohere ReAct Agent to answer questions over the income statement and balance sheet from Apple's SEC10K 2020 form.
+ tags:
+ - agents
+ - oss
+ authors:
+ - shaan@cohere.com
+
+- title: Agentic RAG for PDFs with mixed data
+ path: notebooks/agents/agentic-RAG/agentic_rag_langchain.ipynb
+ slug: agentic-rag-mixed-data
+ blurb: A walkthrough of how to use Langchain cohere_react_agent to run RAG as an agent tool to handle PDFs with mixed table and text data.
+ tags:
+ - agents
+ authors:
+ - shaan@cohere.com
+
+- title: SQL Agent
+ path: notebooks/agents/sql_agent/sql_agent.ipynb
+ slug: sql-agent
+ blurb: In this notebook we explore how to setup a Cohere ReAct Agent to answer questions over SQL Databases using Langchain's SQLDBToolkit.
+ tags:
+ - agents
+ authors:
+ - shaan@cohere.com
+
+- title: Financial CSV Agent with Native Multi-Step Cohere API
+ path: notebooks/agents/financial-csv-agent/financial_csv_publication_native.ipynb
+ slug: csv-agent-native-api
+ blurb: This notebook demonstrates how to setup a Cohere Native API sequence of tool calls to answer questions over the income statement and balance sheet from Apple's SEC10K 2020 form.
+ tags:
+ - agents
+ authors:
+ - jason@cohere.com
+
+- title:
+ A ReAct Agent with Command R+, Achieving The Same Goals as Adaptive RAG Using
+ Custom Tools
+ path: notebooks/agents/Multi_Step_Tool_Use.ipynb
+ slug: multi-step-custom-tools
+ blurb:
+ A guide to building a ReAct agent with Command R+ for web search, RAG retrieval,
+ and custom tools.
+ tags:
+ - agents
+ - rag
+ publish: false
+
+- title: PDF Extractor with Native Multi Step Tool Use
+ path: notebooks/agents/pdf-extractor/pdf_extractor.ipynb
+ slug: pdf-extractor
+ blurb: How we can leverage agents to extract information from PDFs?
+ tags:
+ - agents
+ authors:
+ - jason@cohere.com
+
+- title: Agentic Multi-Stage RAG with Cohere Tools API
+ path: notebooks/agents/agentic-RAG/agentic_multi_stage_rag_native.ipynb
+ slug: agentic-multi-stage-rag
+ blurb: How to use Agents to improve RAG performance.
+ tags:
+ - agents
+ authors:
+ - jason@cohere.com
+
+- title: Agentic RAG with an Evaluator, Web Search, Human Input, and Python Tool.
+ path: notebooks/agents/agentic-RAG/multi_purpose_agent.ipynb
+ slug: multi-purpose-agent
+ blurb: How to incorporate multiple databases, tools, and human feedback into agentic RAG.
+ tags:
+ - agents
+ authors:
+ - jason@cohere.com
+
+## Other
+- title: Using Redis with Cohere
+ path: notebooks/Cohere_Redis_Guide.ipynb
+ slug: redis-guide
+ blurb:
+ Learn how to use Cohere's text vectorizer with Redis to create a semantic
+ search index.
+ tags:
+ - cloud
+ - search
+ publish: false
+
+- title: Wikipedia Semantic Search with Cohere Embedding Archives
+ path: notebooks/Wikipedia_Semantic_Search_With_Cohere_Embeddings_Archives.ipynb
+ slug: wikipedia-semantic-search
+ blurb: Find relevant Wikipedia passages with semantic search and Cohere embeddings.
+ tags:
+ - search
+
+- title: Semantic Search with Cohere Embed Jobs and Pinecone serverless Solution
+ path: notebooks/Embed_Jobs_Serverless_Pinecone_Semantic_Search.ipynb
+ slug: embed-jobs-serverless-pinecone
+ blurb:
+ Learn how to use Cohere's Embed Jobs and Pinecone's serverless solution to
+ perform semantic search.
+ tags:
+ - cloud
+ - search
+
+- title: Basic RAG
+ path: notebooks/Vanilla_RAG.ipynb
+ slug: basic-rag
+ blurb:
+ RAG boosts the accuracy of language models by combining them with a retrieval
+ system.
+ tags:
+ - rag
+
+- title: End-to-end RAG using Elasticsearch and Cohere
+ path: notebooks/Cohere_Elastic_Guide.ipynb
+ slug: elasticsearch-and-cohere
+ blurb:
+ Learn how to use Cohere and Elastic for semantic search and retrieval-augmented
+ generation.
+ tags:
+ - search
+ - rag
+ - cloud
+
+- title: Semantic Search with Cohere Embed Jobs
+ path: notebooks/Embed_Jobs_Semantic_Search.ipynb
+ slug: embed-jobs
+ blurb: Learn how to use Cohere Embed Jobs to create semantic search functionality.
+ tags:
+ - search
+
+- title: Fueling Generative Content with Keyword Research
+ path: notebooks/guides/Fueling_Generative_Content_with_Keyword_Research.ipynb
+ slug: fueling-generative-content
+ blurb:
+ Enhance content creation with keyword-based topic clusters, generating blog
+ ideas with Cohere's Chat model.
+
+- title: Chunking Strategies
+ path: notebooks/guides/Chunking_strategies.ipynb
+ slug: chunking-strategies
+ blurb: Explore chunking strategies for RAG systems.
+ tags:
+ - rag
+ authors:
+ - anna@cohere.com
+
+- title: Text Classification Using Embeddings
+ path: notebooks/guides/Text_Classification_Using_Embeddings.ipynb
+ slug: text-classification-using-embeddings
+ blurb:
+ Build a text classifier with Cohere embeddings. This notebook shows you how
+ to train a sentiment analysis model with a small dataset.
+
+- title: Recipes for Better Meeting Notes
+ path: notebooks/guides/Recipes_for_better_meeting_notes_summary.ipynb
+ slug: recipes-for-meeting-notes
+ blurb:
+ Learn how to use Command to auto-generate meeting summaries, focusing on
+ action items, speaker perspectives, and specific topics.
+ tags:
+ - summarization
+ publish: false
+
+- title: Analysis of Form 10-K/10-Q Using Cohere and RAG
+ path: notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb
+ slug: analysis-of-financial-forms
+ blurb:
+ Jumpstart financial analysis of 10-Ks or 10-Qs with Cohere's Command model
+ and LlamaIndex tooling.
+ tags:
+ - summarization
+ authors:
+ - alexandrebarbet@cohere.com
+
+- title:
+ Question Answering using LangChain and Cohere's Generate and Embedding Models
+ from SageMaker JumpStart
+ path: notebooks/guides/QuestionAnwering_Cohere_SagemakerJumpstart.ipynb
+ slug: sagemaker-question-answering
+ blurb: Use Cohere's LLMs with SageMaker to build a question-answering app.
+ tags:
+ - cloud
+ - oss
+ publish: false
+
+- title: Article Recommender with Text Embedding Classification Extraction
+ path: notebooks/guides/Article_Recommender_with_Text_Embedding_Classification_Extraction.ipynb
+ slug: article-recommender-with-text-embeddings
+ blurb:
+ Improve news article recommendations with embeddings, text classification,
+ and keyword extraction.
+
+- title: Migrating Monolithic Prompts to Command-R with RAG
+ path: notebooks/guides/Migrating_Monolithic_Prompts_to_Command_R_with_RAG.ipynb
+ slug: migrating-prompts
+ blurb:
+ Command-R simplifies prompt migration to RAG, reducing hallucination and
+ improving conciseness and grounding.
+ tags:
+ - rag
+
+- title: Basic Semantic Search
+ path: notebooks/guides/Basic_Semantic_Search.ipynb
+ slug: basic-semantic-search
+ blurb: Learn how to build a simple semantic search engine using sentence embeddings.
+ tags:
+ - search
+
+- title: Using Generation Models for Summarization
+ path: notebooks/guides/Basic_Summarization_Notebook.ipynb
+ slug: basic-summarization
+ blurb:
+ Learn how to use generation models to summarize text with Cohere's Python
+ SDK.
+ tags:
+ - summarization
+ publish: false
+
+- title: Long Form General Strategies
+ path: notebooks/guides/Long_form_General_Strategies.ipynb
+ slug: long-form-general-strategies
+ blurb: Techniques to address lengthy documents exceeding the context window of LLMs.
+ tags:
+ - summarization
+ authors:
+ - anna@cohere.com
+
+- title: Summarization Evals
+ path: notebooks/guides/Summarization_Evals.ipynb
+ slug: summarization-evals
+ blurb:
+ This cookbook demonstrates an approach to evaluating summarization tasks
+ using LLM evaluation.
+ tags:
+ - summarization
+
+- title: Multilingual Search with Cohere and Langchain
+ path: notebooks/guides/Multilingual_Search_with_Cohere_and_Langchain.ipynb
+ slug: multilingual-search
+ blurb: Multilingual search with Cohere and Langchain.
+ tags:
+ - search
+ - oss
+
+- title: Advanced Document Parsing For Enterprises
+ path: notebooks/guides/Document_Parsing_For_Enterprises.ipynb
+ slug: document-parsing-for-enterprises
+ blurb: Learn how to parse PDFs into text with a real-world example.
+ authors:
+ - giannis@cohere.com
+ - justin@cohere.com
+
+- title: Summarizing Meeting Notes
+ path: notebooks/guides/Meeting_Summaries_General_&_LangChain.ipynb
+ slug: summarizing-meeting-notes
+ blurb:
+ Learn how to use Cohere's Command-R model for summarization with this comprehensive
+ guide.
+ tags:
+ - summarization
+ publish: false
+
+- title: Generating Stories and Story ideas with Cohere and Stable Diffusion
+ path: notebooks/guides/Generating_stories_with_LLMs_and_Stable_Diffusion.ipynb
+ slug: generating-stories
+ blurb: This notebook teaches users how to generate sci-fi stories using AI.
+ publish: false
+
+- title: Wikipedia Semantic Search with Cohere + Weaviate
+ path: notebooks/guides/Wikipedia_search_demo_cohere_weaviate.ipynb
+ slug: wikipedia-search-with-weaviate
+ blurb:
+ Search 10 million Wikipedia vectors with Cohere's multilingual model and
+ Weaviate's public dataset.
+ tags:
+ - search
+
+- title: Brainstorming Story Ideas with Cohere and Stable Diffusion
+ path: notebooks/guides/Brainstorming_Story_Ideas_with_Cohere_and_Stable_Diffusion.ipynb
+ slug: brainstorming-story-ideas
+ blurb:
+ Two hackers, suspected of stealing Bitcoin, must prove their innocence to
+ a cyber intelligence agent.
+ publish: false
+
+- title: RAG With Chat Embed and Rerank via Pinecone
+ path: notebooks/guides/RAG_with_Chat_Embed_and_Rerank_via_Pinecone.ipynb
+ slug: rag-with-chat-embed
+ blurb:
+ This notebook shows how to build a RAG-powered chatbot with Cohere's Chat
+ endpoint.
+ tags:
+ - rag
+
+- title: Creating a QA Bot From Technical Documentation
+ path: notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb
+ slug: creating-a-qa-bot
+ blurb:
+ Create a chatbot that answers user questions based on technical documentation
+ using Cohere embeddings and LlamaIndex.
+ tags:
+ - oss
+ - search
+ - rag
+
+- title: "Pondr, Fostering Connection through Good Conversation"
+ path: notebooks/guides/Pondr_Question_Generation.ipynb
+ slug: pondr
+ blurb:
+ Learn how to create Pondr, a game that fosters connections and meaningful
+ conversations with friends and strangers.
+
+- title: Demo of Rerank
+ path: notebooks/guides/rerank-demo.ipynb
+ slug: rerank-demo
+ blurb:
+ Improve search results with Cohere's Relevance Endpoint, which reranks documents
+ for better accuracy.
+ tags:
+ - search
+
+- title: Hello World! Meet Language AI
+ path: notebooks/guides/Hello_World_Meet_Language_AI.ipynb
+ slug: hello-world-meet-ai
+ blurb: General quickstart with basic instructions on how to get started with generative AI.
+
+- title: Entity Extraction with Generative Models
+ path: notebooks/guides/Entity_Extrcation_with_Generative_Language_Models.ipynb
+ slug: entity-extraction
+ blurb:
+ Learn how to extract named entities from text using Cohere's generative models
+ and Python.
+ publish: false
+
+- title: Three Ways to Build a Text Classifier with Cohere.ipynb
+ path: notebooks/guides/Three_Ways_to_Build_a_Text_Classifier_with_Cohere.ipynb
+ slug: ways-to-build-a-text-classifier-with-cohere
+ blurb:
+ Learn three ways to build a text classifier with Cohere's API, from few-shot
+ to finetuning.
+ publish: false
+
+- title: Topic Modeling AI Papers
+ path: notebooks/guides/Topic_Modeling_AI_Papers.ipynb
+ slug: topic-modeling-ai-papers
+ blurb: Learn how to build a topic modeling pipeline in Python.
+
+- title: Grounded Summarization Using Command R
+ path: notebooks/guides/Grounded_summarisation_using_Command_R.ipynb
+ slug: grounded-summarization
+ blurb:
+ Learn how to summarise long documents with citations, reducing cost and improving
+ latency.
+ tags:
+ - summarization
+
+- title: Deep Dive Into RAG Evaluation
+ path: notebooks/guides/Deep_dive_into_RAG_evaluation.ipynb
+ slug: rag-evaluation-deep-dive
+ blurb: Learn how to evaluate RAG models.
+ tags:
+ - rag
+ authors:
+ - marco@cohere.com
+ - aal@cohere.com
+
+- title: Analyzing Hacker News with Six Language Understanding Methods
+ path: notebooks/guides/Analyzing_Hacker_News_with_Six_Language_Understanding_Methods.ipynb
+ slug: analyzing-hacker-news
+ blurb: Learn how to analyze textual data using Cohere's tools.
diff --git a/scripts/build-cookbooks/templates/cookbook.md b/scripts/build-cookbooks/templates/cookbook.md
new file mode 100644
index 000000000..8dadf12cd
--- /dev/null
+++ b/scripts/build-cookbooks/templates/cookbook.md
@@ -0,0 +1,28 @@
+---
+title: {{ title }}
+slug: {{ slug }}
+
+description: "{{ description }}"
+image: "{{ image }}"
+keywords: "{{ keywords }}"
+---
+
+import { AuthorsContainer } from "../../components/authors-container";
+import { CookbookHeader } from "../../components/cookbook-header";
+
+{% if authors -%}
+
| - | instruction | -eval_metadata | -objective | -transcript | -prompt | -transcript_token_len | -completion | -format_score | -length_score | -completeness_score | -correctness_score | -conciseness_score | -
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -Summarize the meeting based on the transcript.... | -\{'format': 'paragraphs', 'min_length': 10, 'ma... | -general_summarization | -PhD F: As opposed to the rest of us \nPhD D: W... | -## meeting transcript\nPhD F: As opposed to th... | -1378 | -PhD D is transcribing recorded sessions to loc... | -1 | -1 | -0.8 | -1.0 | -0.8 | -
| 1 | -Summarize the meeting based on the transcript.... | -\{'format': 'paragraphs', 'min_length': 50, 'ma... | -general_summarization | -Lynne Neagle AM: Thank you very much And the n... | -## meeting transcript\nLynne Neagle AM: Thank ... | -1649 | -The discussion focused on the impact of COVID1... | -1 | -1 | -0.8 | -1.0 | -0.8 | -
| 2 | -Summarize the meeting based on the transcript.... | -\{'format': 'bullets', 'number': 3, 'min_length... | -general_summarization | -Industrial Designer: Yep So we are to mainly d... | -## meeting transcript\nIndustrial Designer: Ye... | -1100 | -- The team is designing a remote control with ... | -1 | -0 | -0.8 | -1.0 | -0.8 | -
| 3 | -Summarize the meeting based on the transcript.... | -\{'format': 'bullets', 'number': 2, 'min_length... | -general_summarization | -Industrial Designer: Mm I think one of the ver... | -## meeting transcript\nIndustrial Designer: Mm... | -2618 | -- The team discusses the target demographic fo... | -1 | -1 | -0.8 | -1.0 | -0.8 | -
| 4 | -What are the follow-up items based on the meet... | -\{'format': 'bullets', 'number': 3, 'min_length... | -action_items | -Marketing: so a lot of people have to be able ... | -## meeting transcript\nMarketing: so a lot of ... | -2286 | -- Investigate how the remote will interact wit... | -1 | -1 | -0.8 | -1.0 | -0.8 | -
| 5 | -What are the follow-up items based on the meet... | -\{'format': 'bullets', 'number': 2, 'min_length... | -action_items | -Project Manager: Alright So finance And we wil... | -## meeting transcript\nProject Manager: Alrigh... | -1965 | -- The project manager will send the updated de... | -1 | -1 | -0.8 | -1.0 | -0.8 | -
| + | instruction | +eval_metadata | +objective | +transcript | +prompt | +transcript_token_len | +completion | +format_score | +length_score | +completeness_score | +correctness_score | +conciseness_score | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | +Summarize the meeting based on the transcript.... | +{'format': 'paragraphs', 'min_length': 10, 'ma... | +general_summarization | +PhD F: As opposed to the rest of us \nPhD D: W... | +## meeting transcript\nPhD F: As opposed to th... | +1378 | +PhD student D presented their method for detec... | +1 | +1 | +0.8 | +1.0 | +0.8 | +
| 1 | +Summarize the meeting based on the transcript.... | +{'format': 'paragraphs', 'min_length': 50, 'ma... | +general_summarization | +Lynne Neagle AM: Thank you very much And the n... | +## meeting transcript\nLynne Neagle AM: Thank ... | +1649 | +The meeting between Welsh politicians and offi... | +1 | +1 | +0.8 | +1.0 | +0.8 | +
| 2 | +Summarize the meeting based on the transcript.... | +{'format': 'bullets', 'number': 3, 'min_length... | +general_summarization | +Industrial Designer: Yep So we are to mainly d... | +## meeting transcript\nIndustrial Designer: Ye... | +1100 | +- The team discusses the design of a remote co... | +1 | +0 | +0.8 | +1.0 | +0.8 | +
| 3 | +Summarize the meeting based on the transcript.... | +{'format': 'bullets', 'number': 2, 'min_length... | +general_summarization | +Industrial Designer: Mm I think one of the ver... | +## meeting transcript\nIndustrial Designer: Mm... | +2618 | +- The team discusses the target demographic fo... | +1 | +1 | +0.8 | +1.0 | +0.8 | +
| 4 | +What are the follow-up items based on the meet... | +{'format': 'bullets', 'number': 3, 'min_length... | +action_items | +Marketing: so a lot of people have to be able ... | +## meeting transcript\nMarketing: so a lot of ... | +2286 | +- Investigate remote's compatibility with vari... | +1 | +1 | +0.8 | +1.0 | +0.8 | +
| 5 | +What are the follow-up items based on the meet... | +{'format': 'bullets', 'number': 2, 'min_length... | +action_items | +Project Manager: Alright So finance And we wil... | +## meeting transcript\nProject Manager: Alrigh... | +1965 | +- The project manager will send the updated de... | +1 | +1 | +0.8 | +1.0 | +0.8 | +
+
+
diff --git a/fern/pages/cookbooks/text-classification-using-embeddings.mdx b/fern/pages/cookbooks/text-classification-using-embeddings.mdx
index 351b4f139..ae9f88ee8 100644
--- a/fern/pages/cookbooks/text-classification-using-embeddings.mdx
+++ b/fern/pages/cookbooks/text-classification-using-embeddings.mdx
@@ -3,22 +3,21 @@ title: Text Classification Using Embeddings
slug: /page/text-classification-using-embeddings
description: "This page discusses the creation of a text classification model using word vector embeddings."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, text classification, classification models, word vector embeddings"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-
+
The example classification task here will be sentiment analysis of film reviews. We'll train a simple classifier to detect whether a film review is negative (class 0) or positive (class 1).
@@ -29,95 +28,113 @@ We'll go through the following steps:
3. Train a classifier using the training set
4. Evaluate the performance of the classifier on the testing set
+
+```python
+# Let's first install Cohere's python SDK
+# TODO: upgrade to "cohere>5"!pip install "cohere<5" scikit-learn
+```
+
If you're running an older version of the SDK you'll want to upgrade it, like this:
-```python PYTHON
+
+```python
#!pip install --upgrade cohere
```
## 1. Get the dataset
-```python PYTHON
+
+```python
import cohere
from sklearn.model_selection import train_test_split
import pandas as pd
pd.set_option('display.max_colwidth', None)
+# Get the SST2 training and test sets
df = pd.read_csv('https://github.com/clairett/pytorch-sentiment-classification/raw/master/data/SST2/train.tsv', delimiter='\t', header=None)
```
-```python PYTHON
+
+```python
+# Let's glance at the dataset
df.head()
```
+
+
+
-
-
-
+
+
+
+
+
+
We'll only use a subset of the training and testing datasets in this example. We'll only use 500 examples since this is a toy example. You'll want to increase the number to get better performance and evaluation.
The `train_test_split` method splits arrays or matrices into random train and test subsets.
-```python PYTHON
+
+```python
+# Set the number of examples from the dataset
num_examples = 500
+# Create a dataframe that
df_sample = df.sample(num_examples)
+# Split into training and testing sets
sentences_train, sentences_test, labels_train, labels_test = train_test_split(
list(df_sample[0]), list(df_sample[1]), test_size=0.25, random_state=0)
+# The embeddings endpoint can take up to 96 texts, so we'll have to truncate
+# sentences_train, sentences_test, labels_train, and labels_test.
sentences_train = sentences_train[:95]
sentences_test = sentences_test[:95]
@@ -127,78 +144,94 @@ labels_test = labels_test[:95]
```
## 2. Set up the Cohere client and get the embeddings of the reviews
+We're now ready to retrieve the embeddings from the API. You'll need your API key for this next cell. [Sign up to Cohere](https://os.cohere.ai/) and get one if you haven't yet.
-We're now ready to retrieve the embeddings from the API. You'll need your API key for this next cell. [Sign up to Cohere](https://dashboard.cohere.com/) and get one if you haven't yet.
-```python PYTHON
+```python
+# Add the model name, API key, URL, etc.
model_name = "embed-english-v3.0"
api_key = ""
+# Here, we're setting up the data objects we'll pass to the embeds endpoint.
input_type = "classification"
+# Create and retrieve a Cohere API key from dashboard.cohere.ai
co = cohere.Client(api_key)
```
-```python PYTHON
+
+```python
+# Embed the training set
embeddings_train = co.embed(texts=sentences_train,
model=model_name,
input_type=input_type
).embeddings
+# Embed the testing set
embeddings_test = co.embed(texts=sentences_test,
model=model_name,
input_type=input_type
).embeddings
+# Here we are using the endpoint co.embed()
```
Note that the ordering of the arguments is important. If you put `input_type` in before `model_name`, you'll get an error.
-We now have two sets of embeddings, `embeddings_train` contains the embeddings of the training sentences while `embeddings_test` contains the embeddings of the testing sentences.
+We now have two sets of embeddings, `embeddings_train` contains the embeddings of the training sentences while `embeddings_test` contains the embeddings of the testing sentences.
+
+Curious what an embedding looks like? we can print it:
-Curious what an embedding looks like? We can print it:
-```python PYTHON
+```python
print(f"Review text: {sentences_train[0]}")
print(f"Embedding vector: {embeddings_train[0][:10]}")
```
-```
-Review text: the script was reportedly rewritten a dozen times either 11 times too many or else too few
-Embedding vector: [1.1531117, -0.8543223, -1.2496399, -0.28317127, -0.75870246, 0.5373464, 0.63233083, 0.5766576, 1.8336298, 0.44203663]
-```
+ Review text: the script was reportedly rewritten a dozen times either 11 times too many or else too few
+ Embedding vector: [1.1531117, -0.8543223, -1.2496399, -0.28317127, -0.75870246, 0.5373464, 0.63233083, 0.5766576, 1.8336298, 0.44203663]
+
## 3. Train a classifier using the training set
+Now that we have the embedding we can train our classifier. We'll use an SVM from sklearn.
-Now that we have the embedding, we can train our classifier. We'll use an SVM from sklearn.
-```python PYTHON
+```python
+# import SVM classifier code
from sklearn.svm import SVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
-svm_classifier = make_pipeline(StandardScaler(), SVC(class_weight='balanced'))
+# Initialize a support vector machine, with class_weight='balanced' because
+# our training set has roughly an equal amount of positive and negative
+# sentiment sentences
+svm_classifier = make_pipeline(StandardScaler(), SVC(class_weight='balanced'))
+# fit the support vector machine
svm_classifier.fit(embeddings_train, labels_train)
```
-```
-Pipeline(steps=[('standardscaler', StandardScaler()),
- ('svc', SVC(class_weight='balanced'))])
-```
+
+
+
+ Pipeline(steps=[('standardscaler', StandardScaler()),
+ ('svc', SVC(class_weight='balanced'))])
+
+
## 4. Evaluate the performance of the classifier on the testing set
-```python PYTHON
+
+```python
+# get the score from the test set, and print it out to screen!
score = svm_classifier.score(embeddings_test, labels_test)
print(f"Validation accuracy on is {100*score}%!")
```
-```
-Validation accuracy on Large is 91.2%!
-```
+ Validation accuracy on Large is 91.2%!
+
You may get a slightly different number when you run this code.
diff --git a/fern/pages/cookbooks/topic-modeling-ai-papers.mdx b/fern/pages/cookbooks/topic-modeling-ai-papers.mdx
index d661132d2..b3fb83e46 100644
--- a/fern/pages/cookbooks/topic-modeling-ai-papers.mdx
+++ b/fern/pages/cookbooks/topic-modeling-ai-papers.mdx
@@ -1,43 +1,55 @@
---
-title: Topic Modeling System for AI Papers
+title: Topic Modeling AI Papers
slug: /page/topic-modeling-ai-papers
description: "This page discusses how to create a topic-modeling system for papers focused on AI papers."
-image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
+image: "../../assets/images/f1cc130-cohere_meta_image.jpg"
keywords: "Cohere, topic modeling, automated science"
---
import { AuthorsContainer } from "../../components/authors-container";
import { CookbookHeader } from "../../components/cookbook-header";
-| - | 0 | -1 | -
|---|---|---|
| 0 | -- a stirring , funny and finally transporting re imagining of beauty and - the beast and 1930s horror films - | -1 | -
| 1 | -- apparently reassembled from the cutting room floor of any given - daytime soap - | -0 | -
| 2 | -- they presume their audience wo n't sit still for a sociology lesson , - however entertainingly presented , so they trot out the conventional - science fiction elements of bug eyed monsters and futuristic women in - skimpy clothes - | -0 | -
| 3 | -- this is a visually stunning rumination on love , memory , history and - the war between art and commerce - | -1 | -
| 4 | -- jonathan parker 's bartleby should have been the be all end all of the - modern office anomie films - | -1 | -
| + | 0 | +1 | +
|---|---|---|
| 0 | +a stirring , funny and finally transporting re imagining of beauty and the beast and 1930s horror films | +1 | +
| 1 | +apparently reassembled from the cutting room floor of any given daytime soap | +0 | +
| 2 | +they presume their audience wo n't sit still for a sociology lesson , however entertainingly presented , so they trot out the conventional science fiction elements of bug eyed monsters and futuristic women in skimpy clothes | +0 | +
| 3 | +this is a visually stunning rumination on love , memory , history and the war between art and commerce | +1 | +
| 4 | +jonathan parker 's bartleby should have been the be all end all of the modern office anomie films | +1 | +