Skip to content

Commit 944ae47

Browse files
authored
[ChatQnA] Fix the service connection issue on GPU and modify the emb backend (#1059)
Signed-off-by: Wang, Kai Lawrence <kai.lawrence.wang@intel.com>
1 parent 2d9aeb3 commit 944ae47

File tree

2 files changed

+17
-20
lines changed

2 files changed

+17
-20
lines changed

ChatQnA/docker_compose/nvidia/gpu/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ To construct the Mega Service, we utilize the [GenAIComps](https://github.com/op
9797
git clone https://github.com/opea-project/GenAIExamples.git
9898
cd GenAIExamples/ChatQnA
9999
docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
100-
cd ../../..
100+
cd ../..
101101
```
102102

103103
### 5. Build UI Docker Image
@@ -107,7 +107,7 @@ Construct the frontend Docker image using the command below:
107107
```bash
108108
cd GenAIExamples/ChatQnA/ui
109109
docker build --no-cache -t opea/chatqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
110-
cd ../../../..
110+
cd ../../../
111111
```
112112

113113
### 6. Build React UI Docker Image (Optional)
@@ -117,7 +117,7 @@ Construct the frontend Docker image using the command below:
117117
```bash
118118
cd GenAIExamples/ChatQnA/ui
119119
docker build --no-cache -t opea/chatqna-react-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile.react .
120-
cd ../../../..
120+
cd ../../..
121121
```
122122

123123
### 7. Build Nginx Docker Image

ChatQnA/docker_compose/nvidia/gpu/compose.yaml

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,10 @@ services:
2020
no_proxy: ${no_proxy}
2121
http_proxy: ${http_proxy}
2222
https_proxy: ${https_proxy}
23-
REDIS_URL: ${REDIS_URL}
24-
REDIS_HOST: ${REDIS_HOST}
23+
REDIS_URL: redis://redis-vector-db:6379
24+
REDIS_HOST: redis-vector-db
2525
INDEX_NAME: ${INDEX_NAME}
26-
TEI_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
26+
TEI_ENDPOINT: http://tei-embedding-service:80
2727
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
2828
tei-embedding-service:
2929
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
@@ -39,13 +39,6 @@ services:
3939
http_proxy: ${http_proxy}
4040
https_proxy: ${https_proxy}
4141
command: --model-id ${EMBEDDING_MODEL_ID} --auto-truncate
42-
deploy:
43-
resources:
44-
reservations:
45-
devices:
46-
- driver: nvidia
47-
count: 1
48-
capabilities: [gpu]
4942
retriever:
5043
image: ${REGISTRY:-opea}/retriever-redis:${TAG:-latest}
5144
container_name: retriever-redis-server
@@ -58,12 +51,13 @@ services:
5851
no_proxy: ${no_proxy}
5952
http_proxy: ${http_proxy}
6053
https_proxy: ${https_proxy}
61-
REDIS_URL: ${REDIS_URL}
54+
REDIS_URL: redis://redis-vector-db:6379
55+
REDIS_HOST: redis-vector-db
6256
INDEX_NAME: ${INDEX_NAME}
6357
TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
6458
restart: unless-stopped
6559
tei-reranking-service:
66-
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
60+
image: ghcr.io/huggingface/text-embeddings-inference:1.5
6761
container_name: tei-reranking-server
6862
ports:
6963
- "8808:80"
@@ -123,11 +117,14 @@ services:
123117
- no_proxy=${no_proxy}
124118
- https_proxy=${https_proxy}
125119
- http_proxy=${http_proxy}
126-
- MEGA_SERVICE_HOST_IP=${MEGA_SERVICE_HOST_IP}
127-
- EMBEDDING_SERVICE_HOST_IP=${EMBEDDING_SERVICE_HOST_IP}
128-
- RETRIEVER_SERVICE_HOST_IP=${RETRIEVER_SERVICE_HOST_IP}
129-
- RERANK_SERVICE_HOST_IP=${RERANK_SERVICE_HOST_IP}
130-
- LLM_SERVICE_HOST_IP=${LLM_SERVICE_HOST_IP}
120+
- MEGA_SERVICE_HOST_IP=chaqna-backend-server
121+
- EMBEDDING_SERVER_HOST_IP=tei-embedding-service
122+
- EMBEDDING_SERVER_PORT=${EMBEDDING_SERVER_PORT:-80}
123+
- RETRIEVER_SERVICE_HOST_IP=retriever
124+
- RERANK_SERVER_HOST_IP=tei-reranking-service
125+
- RERANK_SERVER_PORT=${RERANK_SERVER_PORT:-80}
126+
- LLM_SERVER_HOST_IP=tgi-service
127+
- LLM_SERVER_PORT=${LLM_SERVER_PORT:-80}
131128
ipc: host
132129
restart: always
133130
chaqna-ui-server:

0 commit comments

Comments
 (0)