Skip to content

Commit dbf4ba0

Browse files
yinghu5pre-commit-ci[bot]
yinghu5
and
pre-commit-ci[bot]
authored
Update AgentQnA README.md for refactor doc structure (#1146)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 4f96d9e commit dbf4ba0

File tree

2 files changed

+204
-2
lines changed

2 files changed

+204
-2
lines changed

‎AgentQnA/docker_compose/intel/cpu/xeon/README.md

Lines changed: 99 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,100 @@
1-
# Deployment on Xeon
1+
# Single node on-prem deployment with Docker Compose on Xeon Scalable processors
22

3-
We deploy the retrieval tool on Xeon. For LLMs, we support OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md).
3+
This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Xeon. For LLMs, we use OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md).
4+
5+
## Deployment with docker
6+
7+
1. First, clone this repo.
8+
```
9+
export WORKDIR=<your-work-directory>
10+
cd $WORKDIR
11+
git clone https://github.com/opea-project/GenAIExamples.git
12+
```
13+
2. Set up environment for this example </br>
14+
15+
```
16+
# Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP"
17+
export host_ip=$(hostname -I | awk '{print $1}')
18+
# if you are in a proxy environment, also set the proxy-related environment variables
19+
export http_proxy="Your_HTTP_Proxy"
20+
export https_proxy="Your_HTTPs_Proxy"
21+
# Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
22+
export no_proxy="Your_No_Proxy"
23+
24+
export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/
25+
#OPANAI_API_KEY if you want to use OpenAI models
26+
export OPENAI_API_KEY=<your-openai-key>
27+
```
28+
29+
3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service)
30+
31+
First, launch the mega-service.
32+
33+
```
34+
cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool
35+
bash launch_retrieval_tool.sh
36+
```
37+
38+
Then, ingest data into the vector database. Here we provide an example. You can ingest your own data.
39+
40+
```
41+
bash run_ingest_data.sh
42+
```
43+
44+
4. Launch Tool service
45+
In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs.
46+
```
47+
docker run -d -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0
48+
```
49+
5. Launch `Agent` service
50+
51+
The configurations of the supervisor agent and the worker agent are defined in the docker-compose yaml file. We currently use openAI GPT-4o-mini as LLM, and llama3.1-70B-instruct (served by TGI-Gaudi) in Gaudi example. To use openai llm, run command below.
52+
53+
```
54+
cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon
55+
bash launch_agent_service_openai.sh
56+
```
57+
58+
6. [Optional] Build `Agent` docker image if pulling images failed.
59+
60+
```
61+
git clone https://github.com/opea-project/GenAIComps.git
62+
cd GenAIComps
63+
docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile .
64+
```
65+
66+
## Validate services
67+
68+
First look at logs of the agent docker containers:
69+
70+
```
71+
# worker agent
72+
docker logs rag-agent-endpoint
73+
```
74+
75+
```
76+
# supervisor agent
77+
docker logs react-agent-endpoint
78+
```
79+
80+
You should see something like "HTTP server setup successful" if the docker containers are started successfully.</p>
81+
82+
Second, validate worker agent:
83+
84+
```
85+
curl http://${host_ip}:9095/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
86+
"query": "Most recent album by Taylor Swift"
87+
}'
88+
```
89+
90+
Third, validate supervisor agent:
91+
92+
```
93+
curl http://${host_ip}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
94+
"query": "Most recent album by Taylor Swift"
95+
}'
96+
```
97+
98+
## How to register your own tools with agent
99+
100+
You can take a look at the tools yaml and python files in this example. For more details, please refer to the "Provide your own tools" section in the instructions [here](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/langchain/README.md).

‎AgentQnA/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Single node on-prem deployment AgentQnA on Gaudi
2+
3+
This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Gaudi using open-source LLMs,
4+
For more details, please refer to the deployment guide [here](../../../../README.md).
5+
6+
## Deployment with docker
7+
8+
1. First, clone this repo.
9+
```
10+
export WORKDIR=<your-work-directory>
11+
cd $WORKDIR
12+
git clone https://github.com/opea-project/GenAIExamples.git
13+
```
14+
2. Set up environment for this example </br>
15+
16+
```
17+
# Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP"
18+
export host_ip=$(hostname -I | awk '{print $1}')
19+
# if you are in a proxy environment, also set the proxy-related environment variables
20+
export http_proxy="Your_HTTP_Proxy"
21+
export https_proxy="Your_HTTPs_Proxy"
22+
# Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
23+
export no_proxy="Your_No_Proxy"
24+
25+
export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/
26+
# for using open-source llms
27+
export HUGGINGFACEHUB_API_TOKEN=<your-HF-token>
28+
# Example export HF_CACHE_DIR=$WORKDIR so that no need to redownload every time
29+
export HF_CACHE_DIR=<directory-where-llms-are-downloaded>
30+
31+
```
32+
33+
3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service)
34+
35+
First, launch the mega-service.
36+
37+
```
38+
cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool
39+
bash launch_retrieval_tool.sh
40+
```
41+
42+
Then, ingest data into the vector database. Here we provide an example. You can ingest your own data.
43+
44+
```
45+
bash run_ingest_data.sh
46+
```
47+
48+
4. Launch Tool service
49+
In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs.
50+
```
51+
docker run -d -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0
52+
```
53+
5. Launch `Agent` service
54+
55+
To use open-source LLMs on Gaudi2, run commands below.
56+
57+
```
58+
cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/hpu/gaudi
59+
bash launch_tgi_gaudi.sh
60+
bash launch_agent_service_tgi_gaudi.sh
61+
```
62+
63+
6. [Optional] Build `Agent` docker image if pulling images failed.
64+
65+
```
66+
git clone https://github.com/opea-project/GenAIComps.git
67+
cd GenAIComps
68+
docker build -t opea/agent-langchain:latest -f comps/agent/langchain/Dockerfile .
69+
```
70+
71+
## Validate services
72+
73+
First look at logs of the agent docker containers:
74+
75+
```
76+
# worker agent
77+
docker logs rag-agent-endpoint
78+
```
79+
80+
```
81+
# supervisor agent
82+
docker logs react-agent-endpoint
83+
```
84+
85+
You should see something like "HTTP server setup successful" if the docker containers are started successfully.</p>
86+
87+
Second, validate worker agent:
88+
89+
```
90+
curl http://${host_ip}:9095/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
91+
"query": "Most recent album by Taylor Swift"
92+
}'
93+
```
94+
95+
Third, validate supervisor agent:
96+
97+
```
98+
curl http://${host_ip}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
99+
"query": "Most recent album by Taylor Swift"
100+
}'
101+
```
102+
103+
## How to register your own tools with agent
104+
105+
You can take a look at the tools yaml and python files in this example. For more details, please refer to the "Provide your own tools" section in the instructions [here](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/langchain/README.md).

0 commit comments

Comments
 (0)