Skip to content

Commit 5638075

Browse files
authored
Add helm deployment instructions for codegen (#1351)
Signed-off-by: Dolpher Du <dolpher.du@intel.com>
1 parent 2311787 commit 5638075

File tree

15 files changed

+73
-1482
lines changed

15 files changed

+73
-1482
lines changed

CodeGen/README.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -145,15 +145,9 @@ docker compose up -d
145145

146146
Refer to the [Xeon Guide](./docker_compose/intel/cpu/xeon/README.md) for more instructions on building docker images from source.
147147

148-
### Deploy CodeGen using Kubernetes
149-
150-
Refer to the [Kubernetes Guide](./kubernetes/intel/README.md) for instructions on deploying CodeGen into Kubernetes on Xeon & Gaudi.
151-
152148
### Deploy CodeGen into Kubernetes using Helm Chart
153149

154-
Install Helm (version >= 3.15) first. Refer to the [Helm Installation Guide](https://helm.sh/docs/intro/install/) for more information.
155-
156-
Refer to the [CodeGen helm chart](https://github.com/opea-project/GenAIInfra/tree/main/helm-charts/codegen/README.md) for instructions on deploying CodeGen into Kubernetes on Xeon & Gaudi.
150+
Refer to the [CodeGen helm chart](./kubernetes/helm/README.md) for instructions on deploying CodeGen into Kubernetes on Xeon & Gaudi.
157151

158152
## Consume CodeGen Service
159153

CodeGen/kubernetes/helm/README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Deploy CodeGen on kubernetes cluster
2+
3+
- You should have Helm (version >= 3.15) installed. Refer to the [Helm Installation Guide](https://helm.sh/docs/intro/install/) for more information.
4+
- For more deploy options, refer to [helm charts README](https://github.com/opea-project/GenAIInfra/tree/main/helm-charts#readme).
5+
6+
## Deploy on Xeon
7+
8+
```
9+
export HFTOKEN="insert-your-huggingface-token-here"
10+
helm install codegen oci://ghcr.io/opea-project/charts/codegen --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
11+
```
12+
13+
## Deploy on Gaudi
14+
15+
```
16+
export HFTOKEN="insert-your-huggingface-token-here"
17+
helm install codegen oci://ghcr.io/opea-project/charts/codegen --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f gaudi-values.yaml
18+
```
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
image:
5+
repository: opea/codegen
6+
tag: "latest"
7+
8+
tgi:
9+
LLM_MODEL_ID: Qwen/Qwen2.5-Coder-7B-Instruct
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
tgi:
5+
accelDevice: "gaudi"
6+
LLM_MODEL_ID: Qwen/Qwen2.5-Coder-7B-Instruct
7+
image:
8+
repository: ghcr.io/huggingface/tgi-gaudi
9+
tag: "2.0.6"
10+
resources:
11+
limits:
12+
habana.ai/gaudi: 1
13+
MAX_INPUT_LENGTH: "1024"
14+
MAX_TOTAL_TOKENS: "2048"
15+
CUDA_GRAPHS: ""
16+
livenessProbe:
17+
initialDelaySeconds: 5
18+
periodSeconds: 5
19+
timeoutSeconds: 1
20+
readinessProbe:
21+
initialDelaySeconds: 5
22+
periodSeconds: 5
23+
timeoutSeconds: 1
24+
startupProbe:
25+
initialDelaySeconds: 5
26+
periodSeconds: 5
27+
timeoutSeconds: 1
28+
failureThreshold: 120

CodeGen/kubernetes/intel/README.md

Lines changed: 0 additions & 48 deletions
This file was deleted.

CodeGen/kubernetes/intel/cpu/xeon/manifest/README_react_ui.md

Lines changed: 0 additions & 38 deletions
This file was deleted.

0 commit comments

Comments
 (0)