Skip to content

Commit b7dfd0f

Browse files
chyundunovDatamonstersChingis Yundunov
authored andcommitted
AgentQnA - add README file for deploy on ROCm (opea-project#1379)
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
1 parent a1aeae3 commit b7dfd0f

File tree

1 file changed

+101
-0
lines changed
  • AgentQnA/docker_compose/amd/gpu/rocm

1 file changed

+101
-0
lines changed
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Single node on-prem deployment with Docker Compose on AMD GPU
2+
3+
This example showcases a hierarchical multi-agent system for question-answering applications. We deploy the example on Xeon. For LLMs, we use OpenAI models via API calls. For instructions on using open-source LLMs, please refer to the deployment guide [here](../../../../README.md).
4+
5+
## Deployment with docker
6+
7+
1. First, clone this repo.
8+
```
9+
export WORKDIR=<your-work-directory>
10+
cd $WORKDIR
11+
git clone https://github.com/opea-project/GenAIExamples.git
12+
```
13+
2. Set up environment for this example </br>
14+
15+
```
16+
# Example: host_ip="192.168.1.1" or export host_ip="External_Public_IP"
17+
export host_ip=$(hostname -I | awk '{print $1}')
18+
# if you are in a proxy environment, also set the proxy-related environment variables
19+
export http_proxy="Your_HTTP_Proxy"
20+
export https_proxy="Your_HTTPs_Proxy"
21+
# Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
22+
export no_proxy="Your_No_Proxy"
23+
24+
export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/
25+
#OPANAI_API_KEY if you want to use OpenAI models
26+
export OPENAI_API_KEY=<your-openai-key>
27+
# Set AMD GPU settings
28+
export AGENTQNA_CARD_ID="card1"
29+
export AGENTQNA_RENDER_ID="renderD136"
30+
```
31+
32+
3. Deploy the retrieval tool (i.e., DocIndexRetriever mega-service)
33+
34+
First, launch the mega-service.
35+
36+
```
37+
cd $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool
38+
bash launch_retrieval_tool.sh
39+
```
40+
41+
Then, ingest data into the vector database. Here we provide an example. You can ingest your own data.
42+
43+
```
44+
bash run_ingest_data.sh
45+
```
46+
47+
4. Launch Tool service
48+
In this example, we will use some of the mock APIs provided in the Meta CRAG KDD Challenge to demonstrate the benefits of gaining additional context from mock knowledge graphs.
49+
```
50+
docker run -d -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0
51+
```
52+
5. Launch `Agent` service
53+
54+
```
55+
cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/amd/gpu/rocm
56+
bash launch_agent_service_tgi_rocm.sh
57+
```
58+
59+
6. [Optional] Build `Agent` docker image if pulling images failed.
60+
61+
```
62+
git clone https://github.com/opea-project/GenAIComps.git
63+
cd GenAIComps
64+
docker build -t opea/agent:latest -f comps/agent/src/Dockerfile .
65+
```
66+
67+
## Validate services
68+
69+
First look at logs of the agent docker containers:
70+
71+
```
72+
# worker agent
73+
docker logs rag-agent-endpoint
74+
```
75+
76+
```
77+
# supervisor agent
78+
docker logs react-agent-endpoint
79+
```
80+
81+
You should see something like "HTTP server setup successful" if the docker containers are started successfully.</p>
82+
83+
Second, validate worker agent:
84+
85+
```
86+
curl http://${host_ip}:9095/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
87+
"query": "Most recent album by Taylor Swift"
88+
}'
89+
```
90+
91+
Third, validate supervisor agent:
92+
93+
```
94+
curl http://${host_ip}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
95+
"query": "Most recent album by Taylor Swift"
96+
}'
97+
```
98+
99+
## How to register your own tools with agent
100+
101+
You can take a look at the tools yaml and python files in this example. For more details, please refer to the "Provide your own tools" section in the instructions [here](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/src/README.md).

0 commit comments

Comments
 (0)