Skip to content

Commit 55585ab

Browse files
ashahbaashahba
authored
Merge branch 'main' into mmqna-audio-query
2 parents ba1fd52 + a50e4e6 commit 55585ab

File tree

10 files changed

+309
-19
lines changed

10 files changed

+309
-19
lines changed

‎DocIndexRetriever/docker_compose/intel/cpu/xeon/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,17 @@ cd GenAIExamples/DocIndexRetriever/intel/cpu/xoen/
6262
docker compose up -d
6363
```
6464

65+
Two types of DocRetriever pipeline are supported now: `DocRetriever with/without Rerank`. And the `DocRetriever without Rerank` pipeline (including Embedding and Retrieval) is offered for customers who expect to handle all retrieved documents by LLM, and require high performance of DocRetriever.
66+
In that case, start Docker Containers with compose_without_rerank.yaml
67+
68+
```bash
69+
export host_ip="YOUR IP ADDR"
70+
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
71+
export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
72+
cd GenAIExamples/DocIndexRetriever/intel/cpu/xoen/
73+
docker compose -f compose_without_rerank.yaml up -d
74+
```
75+
6576
## 4. Validation
6677

6778
Add Knowledge Base via HTTP Links:

‎DocIndexRetriever/docker_compose/intel/cpu/xeon/compose_without_rerank.yaml

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
version: "3.8"
5+
6+
services:
7+
redis-vector-db:
8+
image: redis/redis-stack:7.2.0-v9
9+
container_name: redis-vector-db
10+
ports:
11+
- "6379:6379"
12+
- "8001:8001"
13+
dataprep-redis-service:
14+
image: ${REGISTRY:-opea}/dataprep-redis:${TAG:-latest}
15+
container_name: dataprep-redis-server
16+
depends_on:
17+
- redis-vector-db
18+
ports:
19+
- "6007:6007"
20+
- "6008:6008"
21+
- "6009:6009"
22+
environment:
23+
no_proxy: ${no_proxy}
24+
http_proxy: ${http_proxy}
25+
https_proxy: ${https_proxy}
26+
REDIS_URL: redis://redis-vector-db:6379
27+
REDIS_HOST: redis-vector-db
28+
INDEX_NAME: ${INDEX_NAME:-rag-redis}
29+
TEI_EMBEDDING_ENDPOINT: http://tei-embedding-service:80
30+
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
31+
tei-embedding-service:
32+
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
33+
container_name: tei-embedding-server
34+
ports:
35+
- "6006:80"
36+
volumes:
37+
- "/home/ligang/models:/data"
38+
shm_size: 1g
39+
environment:
40+
no_proxy: ${no_proxy}
41+
http_proxy: ${http_proxy}
42+
https_proxy: ${https_proxy}
43+
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
44+
command: --model-id ${EMBEDDING_MODEL_ID} --auto-truncate
45+
embedding:
46+
image: ${REGISTRY:-opea}/embedding-tei:${TAG:-latest}
47+
container_name: embedding-tei-server
48+
ports:
49+
- "6000:6000"
50+
ipc: host
51+
depends_on:
52+
- tei-embedding-service
53+
environment:
54+
no_proxy: ${no_proxy}
55+
http_proxy: ${http_proxy}
56+
https_proxy: ${https_proxy}
57+
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
58+
TEI_EMBEDDING_ENDPOINT: http://tei-embedding-service:80
59+
restart: unless-stopped
60+
retriever:
61+
image: ${REGISTRY:-opea}/retriever-redis:${TAG:-latest}
62+
container_name: retriever-redis-server
63+
depends_on:
64+
- redis-vector-db
65+
ports:
66+
- "7000:7000"
67+
ipc: host
68+
environment:
69+
no_proxy: ${no_proxy}
70+
http_proxy: ${http_proxy}
71+
https_proxy: ${https_proxy}
72+
REDIS_URL: redis://redis-vector-db:6379
73+
INDEX_NAME: ${INDEX_NAME:-rag-redis}
74+
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
75+
TEI_EMBEDDING_ENDPOINT: http://tei-embedding-service:80
76+
restart: unless-stopped
77+
doc-index-retriever-server:
78+
image: ${REGISTRY:-opea}/doc-index-retriever:${TAG:-latest}
79+
container_name: doc-index-retriever-server
80+
depends_on:
81+
- redis-vector-db
82+
- tei-embedding-service
83+
- embedding
84+
- retriever
85+
ports:
86+
- "8889:8889"
87+
environment:
88+
no_proxy: ${no_proxy}
89+
https_proxy: ${https_proxy}
90+
http_proxy: ${http_proxy}
91+
MEGA_SERVICE_HOST_IP: ${MEGA_SERVICE_HOST_IP:-0.0.0.0}
92+
EMBEDDING_SERVICE_HOST_IP: embedding
93+
EMBEDDING_SERVICE_PORT: ${EMBEDDING_SERVER_PORT:-6000}
94+
RETRIEVER_SERVICE_HOST_IP: retriever
95+
LOGFLAG: ${LOGFLAG}
96+
ipc: host
97+
restart: always
98+
command: --without-rerank
99+
100+
networks:
101+
default:
102+
driver: bridge

‎DocIndexRetriever/retrieval_tool.py

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Copyright (C) 2024 Intel Corporation
22
# SPDX-License-Identifier: Apache-2.0
33

4+
import argparse
45
import asyncio
56
import os
67
from typing import Union
@@ -124,8 +125,37 @@ def start(self):
124125
output_datatype=Union[RerankedDoc, LLMParamsDoc],
125126
)
126127

128+
def add_remote_service_without_rerank(self):
129+
embedding = MicroService(
130+
name="embedding",
131+
host=EMBEDDING_SERVICE_HOST_IP,
132+
port=EMBEDDING_SERVICE_PORT,
133+
endpoint="/v1/embeddings",
134+
use_remote_service=True,
135+
service_type=ServiceType.EMBEDDING,
136+
)
137+
retriever = MicroService(
138+
name="retriever",
139+
host=RETRIEVER_SERVICE_HOST_IP,
140+
port=RETRIEVER_SERVICE_PORT,
141+
endpoint="/v1/retrieval",
142+
use_remote_service=True,
143+
service_type=ServiceType.RETRIEVER,
144+
)
145+
146+
self.megaservice.add(embedding).add(retriever)
147+
self.megaservice.flow_to(embedding, retriever)
148+
127149

128150
if __name__ == "__main__":
151+
parser = argparse.ArgumentParser()
152+
parser.add_argument("--without-rerank", action="store_true")
153+
154+
args = parser.parse_args()
155+
129156
chatqna = RetrievalToolService(port=MEGA_SERVICE_PORT)
130-
chatqna.add_remote_service()
157+
if args.without_rerank:
158+
chatqna.add_remote_service_without_rerank()
159+
else:
160+
chatqna.add_remote_service()
131161
chatqna.start()

‎DocIndexRetriever/tests/test_compose_without_rerank_on_xeon.sh

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
#!/bin/bash
2+
# Copyright (C) 2024 Intel Corporation
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
set -e
6+
IMAGE_REPO=${IMAGE_REPO:-"opea"}
7+
IMAGE_TAG=${IMAGE_TAG:-"latest"}
8+
echo "REGISTRY=IMAGE_REPO=${IMAGE_REPO}"
9+
echo "TAG=IMAGE_TAG=${IMAGE_TAG}"
10+
export REGISTRY=${IMAGE_REPO}
11+
export TAG=${IMAGE_TAG}
12+
13+
WORKPATH=$(dirname "$PWD")
14+
LOG_PATH="$WORKPATH/tests"
15+
ip_address=$(hostname -I | awk '{print $1}')
16+
17+
function build_docker_images() {
18+
echo "Building Docker Images...."
19+
cd $WORKPATH/docker_image_build
20+
if [ ! -d "GenAIComps" ] ; then
21+
echo "Cloning GenAIComps repository"
22+
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
23+
fi
24+
service_list="dataprep-redis embedding-tei retriever-redis doc-index-retriever"
25+
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
26+
27+
docker pull ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
28+
docker pull redis/redis-stack:7.2.0-v9
29+
docker images && sleep 1s
30+
31+
echo "Docker images built!"
32+
}
33+
34+
function start_services() {
35+
echo "Starting Docker Services...."
36+
cd $WORKPATH/docker_compose/intel/cpu/xeon
37+
export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
38+
export TEI_EMBEDDING_ENDPOINT="http://${ip_address}:6006"
39+
export REDIS_URL="redis://${ip_address}:6379"
40+
export INDEX_NAME="rag-redis"
41+
export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
42+
export MEGA_SERVICE_HOST_IP=${ip_address}
43+
export EMBEDDING_SERVICE_HOST_IP=${ip_address}
44+
export RETRIEVER_SERVICE_HOST_IP=${ip_address}
45+
46+
# Start Docker Containers
47+
docker compose -f compose_without_rerank.yaml up -d
48+
sleep 5m
49+
echo "Docker services started!"
50+
}
51+
52+
function validate() {
53+
local CONTENT="$1"
54+
local EXPECTED_RESULT="$2"
55+
local SERVICE_NAME="$3"
56+
57+
if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then
58+
echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT."
59+
echo 0
60+
else
61+
echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT"
62+
echo 1
63+
fi
64+
}
65+
66+
function validate_megaservice() {
67+
echo "===========Ingest data=================="
68+
local CONTENT=$(http_proxy="" curl -X POST "http://${ip_address}:6007/v1/dataprep" \
69+
-H "Content-Type: multipart/form-data" \
70+
-F 'link_list=["https://opea.dev/"]')
71+
local EXIT_CODE=$(validate "$CONTENT" "Data preparation succeeded" "dataprep-redis-service-xeon")
72+
echo "$EXIT_CODE"
73+
local EXIT_CODE="${EXIT_CODE:0-1}"
74+
echo "return value is $EXIT_CODE"
75+
if [ "$EXIT_CODE" == "1" ]; then
76+
docker logs dataprep-redis-server | tee -a ${LOG_PATH}/dataprep-redis-service-xeon.log
77+
return 1
78+
fi
79+
80+
# Curl the Mega Service
81+
echo "================Testing retriever service: Text Request ================"
82+
cd $WORKPATH/tests
83+
local CONTENT=$(http_proxy="" curl http://${ip_address}:8889/v1/retrievaltool -X POST -H "Content-Type: application/json" -d '{
84+
"text": "Explain the OPEA project?"
85+
}')
86+
# local CONTENT=$(python test.py --host_ip ${ip_address} --request_type text)
87+
local EXIT_CODE=$(validate "$CONTENT" "OPEA" "doc-index-retriever-service-xeon")
88+
echo "$EXIT_CODE"
89+
local EXIT_CODE="${EXIT_CODE:0-1}"
90+
echo "return value is $EXIT_CODE"
91+
if [ "$EXIT_CODE" == "1" ]; then
92+
echo "=============Embedding container log=================="
93+
docker logs embedding-tei-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
94+
echo "=============Retriever container log=================="
95+
docker logs retriever-redis-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
96+
echo "=============Doc-index-retriever container log=================="
97+
docker logs doc-index-retriever-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
98+
exit 1
99+
fi
100+
101+
echo "================Testing retriever service: ChatCompletion Request================"
102+
cd $WORKPATH/tests
103+
local CONTENT=$(python test.py --host_ip ${ip_address} --request_type chat_completion)
104+
local EXIT_CODE=$(validate "$CONTENT" "OPEA" "doc-index-retriever-service-xeon")
105+
echo "$EXIT_CODE"
106+
local EXIT_CODE="${EXIT_CODE:0-1}"
107+
echo "return value is $EXIT_CODE"
108+
if [ "$EXIT_CODE" == "1" ]; then
109+
echo "=============Embedding container log=================="
110+
docker logs embedding-tei-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
111+
echo "=============Retriever container log=================="
112+
docker logs retriever-redis-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
113+
echo "=============Doc-index-retriever container log=================="
114+
docker logs doc-index-retriever-server | tee -a ${LOG_PATH}/doc-index-retriever-service-xeon.log
115+
exit 1
116+
fi
117+
}
118+
119+
function stop_docker() {
120+
cd $WORKPATH/docker_compose/intel/cpu/xeon
121+
container_list=$(cat compose.yaml | grep container_name | cut -d':' -f2)
122+
for container_name in $container_list; do
123+
cid=$(docker ps -aq --filter "name=$container_name")
124+
echo "Stopping container $container_name"
125+
if [[ ! -z "$cid" ]]; then docker rm $cid -f && sleep 1s; fi
126+
done
127+
}
128+
129+
function main() {
130+
131+
stop_docker
132+
build_docker_images
133+
echo "Dump current docker ps"
134+
docker ps
135+
start_time=$(date +%s)
136+
start_services
137+
end_time=$(date +%s)
138+
duration=$((end_time-start_time))
139+
echo "Mega service start duration is $duration s"
140+
validate_megaservice
141+
142+
stop_docker
143+
echo y | docker system prune
144+
145+
}
146+
147+
main

‎DocSum/docker_compose/intel/cpu/xeon/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -67,22 +67,22 @@ docker build -t opea/docsum:latest --build-arg https_proxy=$https_proxy --build-
6767

6868
Several UI options are provided. If you need to work with multimedia documents, .doc, or .pdf files, suggested to use Gradio UI.
6969

70-
#### Svelte UI
70+
#### Gradio UI
7171

72-
Build the frontend Docker image via below command:
72+
Build the Gradio UI frontend Docker image using the following command:
7373

7474
```bash
7575
cd GenAIExamples/DocSum/ui
76-
docker build -t opea/docsum-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
76+
docker build -t opea/docsum-gradio-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile.gradio .
7777
```
7878

79-
#### Gradio UI
79+
#### Svelte UI
8080

81-
Build the Gradio UI frontend Docker image using the following command:
81+
Build the frontend Docker image via below command:
8282

8383
```bash
8484
cd GenAIExamples/DocSum/ui
85-
docker build -t opea/docsum-gradio-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile.gradio .
85+
docker build -t opea/docsum-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
8686
```
8787

8888
#### React UI

‎DocSum/docker_compose/intel/cpu/xeon/compose.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,8 @@ services:
9595
ipc: host
9696
restart: always
9797

98-
docsum-ui:
99-
image: ${REGISTRY:-opea}/docsum-ui:${TAG:-latest}
98+
docsum-gradio-ui:
99+
image: ${REGISTRY:-opea}/docsum-gradio-ui:${TAG:-latest}
100100
container_name: docsum-xeon-ui-server
101101
depends_on:
102102
- docsum-xeon-backend-server

‎DocSum/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -51,22 +51,22 @@ docker build -t opea/docsum:latest --build-arg https_proxy=$https_proxy --build-
5151

5252
Several UI options are provided. If you need to work with multimedia documents, .doc, or .pdf files, suggested to use Gradio UI.
5353

54-
#### Svelte UI
54+
#### Gradio UI
5555

56-
Build the frontend Docker image via below command:
56+
Build the Gradio UI frontend Docker image using the following command:
5757

5858
```bash
5959
cd GenAIExamples/DocSum/ui
60-
docker build -t opea/docsum-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
60+
docker build -t opea/docsum-gradio-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile.gradio .
6161
```
6262

63-
#### Gradio UI
63+
#### Svelte UI
6464

65-
Build the Gradio UI frontend Docker image using the following command:
65+
Build the frontend Docker image via below command:
6666

6767
```bash
6868
cd GenAIExamples/DocSum/ui
69-
docker build -t opea/docsum-gradio-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile.gradio .
69+
docker build -t opea/docsum-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
7070
```
7171

7272
#### React UI

‎DocSum/docker_compose/intel/hpu/gaudi/compose.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,8 @@ services:
108108
ipc: host
109109
restart: always
110110

111-
docsum-ui:
112-
image: ${REGISTRY:-opea}/docsum-ui:${TAG:-latest}
111+
docsum-gradio-ui:
112+
image: ${REGISTRY:-opea}/docsum-gradio-ui:${TAG:-latest}
113113
container_name: docsum-gaudi-ui-server
114114
depends_on:
115115
- docsum-gaudi-backend-server

0 commit comments

Comments
 (0)