Skip to content

Commit b3c405a

Browse files
lvliang-intellvliang-intel
authored
Adapt example code for guardrails refactor (#1360)
Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com>
1 parent 5638075 commit b3c405a

File tree

7 files changed

+15
-15
lines changed

7 files changed

+15
-15
lines changed

‎ChatQnA/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ docker build --no-cache -t opea/dataprep-redis:latest --build-arg https_proxy=$h
9292
To fortify AI initiatives in production, Guardrails microservice can secure model inputs and outputs, building Trustworthy, Safe, and Secure LLM-based Applications.
9393

9494
```bash
95-
docker build -t opea/guardrails-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/guardrails/llama_guard/langchain/Dockerfile .
95+
docker build -t opea/guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/guardrails/src/guardrails/Dockerfile .
9696
```
9797

9898
### 4. Build MegaService Docker Image
@@ -168,7 +168,7 @@ If Conversation React UI is built, you will find one more image:
168168

169169
If Guardrails docker image is built, you will find one more image:
170170

171-
- `opea/guardrails-tgi:latest`
171+
- `opea/guardrails:latest`
172172

173173
## 🚀 Start MicroServices and MegaService
174174

‎ChatQnA/docker_compose/intel/hpu/gaudi/compose_guardrails.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,8 @@ services:
5151
ipc: host
5252
command: --model-id ${GURADRAILS_MODEL_ID} --max-input-length 1024 --max-total-tokens 2048
5353
guardrails:
54-
image: ${REGISTRY:-opea}/guardrails-tgi:${TAG:-latest}
55-
container_name: guardrails-tgi-gaudi-server
54+
image: ${REGISTRY:-opea}/guardrails:${TAG:-latest}
55+
container_name: guardrails-gaudi-server
5656
ports:
5757
- "9090:9090"
5858
ipc: host

‎ChatQnA/docker_image_build/build.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -95,12 +95,12 @@ services:
9595
dockerfile: comps/dataprep/pinecone/langchain/Dockerfile
9696
extends: chatqna
9797
image: ${REGISTRY:-opea}/dataprep-pinecone:${TAG:-latest}
98-
guardrails-tgi:
98+
guardrails:
9999
build:
100100
context: GenAIComps
101-
dockerfile: comps/guardrails/llama_guard/langchain/Dockerfile
101+
dockerfile: comps/guardrails/src/guardrails/Dockerfile
102102
extends: chatqna
103-
image: ${REGISTRY:-opea}/guardrails-tgi:${TAG:-latest}
103+
image: ${REGISTRY:-opea}/guardrails:${TAG:-latest}
104104
vllm:
105105
build:
106106
context: vllm

‎ChatQnA/kubernetes/intel/cpu/xeon/manifest/chatqna-guardrails.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -758,7 +758,7 @@ spec:
758758
runAsUser: 1000
759759
seccompProfile:
760760
type: RuntimeDefault
761-
image: "opea/guardrails-tgi:latest"
761+
image: "opea/guardrails:latest"
762762
imagePullPolicy: Always
763763
ports:
764764
- name: guardrails-usvc

‎ChatQnA/kubernetes/intel/hpu/gaudi/manifest/chatqna-guardrails.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -688,7 +688,7 @@ spec:
688688
runAsUser: 1000
689689
seccompProfile:
690690
type: RuntimeDefault
691-
image: "opea/guardrails-tgi:latest"
691+
image: "opea/guardrails:latest"
692692
imagePullPolicy: Always
693693
ports:
694694
- name: guardrails-usvc

‎ChatQnA/tests/test_compose_guardrails_on_gaudi.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ function build_docker_images() {
1919
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
2020

2121
echo "Build all the images with --no-cache, check docker_image_build.log for details..."
22-
service_list="chatqna-guardrails chatqna-ui dataprep-redis retriever-redis guardrails-tgi nginx"
22+
service_list="chatqna-guardrails chatqna-ui dataprep-redis retriever-redis guardrails nginx"
2323
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
2424

2525
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.6
@@ -136,7 +136,7 @@ function validate_microservices() {
136136
"${ip_address}:9090/v1/guardrails" \
137137
"Violated policies" \
138138
"guardrails" \
139-
"guardrails-tgi-gaudi-server" \
139+
"guardrails-gaudi-server" \
140140
'{"text":"How do you buy a tiger in the US?"}'
141141
}
142142

‎docker_images_list.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,9 +65,9 @@ Take ChatQnA for example. ChatQnA is a chatbot application service based on the
6565
| [opea/finetuning-gaudi](https://hub.docker.com/r/opea/finetuning-gaudi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/finetuning/Dockerfile.intel_hpu) | The docker image exposed the OPEA Fine-tuning microservice for GenAI application use on the Gaudi |
6666
| [opea/gmcrouter](https://hub.docker.com/r/opea/gmcrouter) | [Link](https://github.com/opea-project/GenAIInfra/blob/main/microservices-connector/Dockerfile.manager) | The docker image served as one of key parts of the OPEA GenAI Microservice Connector(GMC) to route the traffic among the microservices defined in GMC |
6767
| [opea/gmcmanager](https://hub.docker.com/r/opea/gmcmanager) | [Link](https://github.com/opea-project/GenAIInfra/blob/main/microservices-connector/Dockerfile.router) | The docker image served as one of key parts of the OPEA GenAI Microservice Connector(GMC) to be controller manager to handle GMC CRD |
68-
| [opea/guardrails-tgi](https://hub.docker.com/r/opea/guardrails-tgi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/llama_guard/langchain/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide content review for GenAI application use |
69-
| [opea/guardrails-toxicity-detection](https://hub.docker.com/r/opea/guardrails-toxicity-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/toxicity_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide toxicity detection for GenAI application use |
70-
| [opea/guardrails-pii-detection](https://hub.docker.com/r/opea/guardrails-pii-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/pii_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide PII detection for GenAI application use |
68+
| [opea/guardrails]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/src/guardrails/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide content review for GenAI application use |
69+
| [opea/guardrails-toxicity-detection](https://hub.docker.com/r/opea/guardrails-toxicity-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/src/toxicity_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide toxicity detection for GenAI application use |
70+
| [opea/guardrails-pii-detection](https://hub.docker.com/r/opea/guardrails-pii-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/src/pii_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide PII detection for GenAI application use |
7171
| [opea/llm-docsum-tgi](https://hub.docker.com/r/opea/llm-docsum-tgi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/summarization/tgi/langchain/Dockerfile) | This docker image is designed to build a document summarization microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a document summary. |
7272
| [opea/llm-faqgen-tgi](https://hub.docker.com/r/opea/llm-faqgen-tgi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/faq-generation/tgi/langchain/Dockerfile) | This docker image is designed to build a frequently asked questions microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a FAQ. |
7373
| [opea/llm-textgen](https://hub.docker.com/r/opea/llm-textgen) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/src/text-generation/Dockerfile) | The docker image exposed the OPEA LLM microservice upon TGI docker image for GenAI application use |
@@ -78,7 +78,7 @@ Take ChatQnA for example. ChatQnA is a chatbot application service based on the
7878
| [opea/lvm-video-llama](https://hub.docker.com/r/opea/lvm-video-llama) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/video-llama/Dockerfile) | The docker image exposed the OPEA microservice running Video-Llama as a large visual model (LVM) for GenAI application use |
7979
| [opea/nginx](https://hub.docker.com/r/opea/nginx) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/3rd_parties/nginx/src/Dockerfile) | The docker image exposed the OPEA nginx microservice for GenAI application use |
8080
| [opea/promptregistry-mongo-server](https://hub.docker.com/r/opea/promptregistry-mongo-server) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/prompt_registry/mongo/Dockerfile) | The docker image exposes the OPEA Prompt Registry microservices which based on MongoDB database, designed to store and retrieve user's preferred prompts |
81-
| [opea/reranking-tei](https://hub.docker.com/r/opea/reranking-tei) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/reranks/src/Dockerfile) | The docker image exposed the OPEA reranking microservice based on tei docker image for GenAI application use |
81+
| [opea/reranking]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/rerankings/src/Dockerfile) | The docker image exposed the OPEA reranking microservice based on tei docker image for GenAI application use |
8282
| [opea/retriever-milvus](https://hub.docker.com/r/opea/retriever-milvus) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/retrievers/milvus/langchain/Dockerfile) | The docker image exposed the OPEA retrieval microservice based on milvus vectordb for GenAI application use |
8383
| [opea/retriever-pathway](https://hub.docker.com/r/opea/retriever-pathway) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/retrievers/pathway/langchain/Dockerfile) | The docker image exposed the OPEA retrieval microservice with pathway for GenAI application use |
8484
| [opea/retriever-pgvector](https://hub.docker.com/r/opea/retriever-pgvector) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/retrievers/pgvector/langchain/Dockerfile) | The docker image exposed the OPEA retrieval microservice based on pgvector vectordb for GenAI application use |

0 commit comments

Comments
 (0)