Skip to content

Commit 6953289

Browse files
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent 2215b5c commit 6953289

File tree

9 files changed

+134
-120
lines changed

9 files changed

+134
-120
lines changed

CodeGen/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@ The workflow falls into the following architecture:
2020

2121
The CodeGen example is implemented using the component-level microservices defined in [GenAIComps](https://github.com/opea-project/GenAIComps). The flow chart below shows the information flow between different microservices for this example.
2222

23-
2423
```mermaid
2524
---
2625
config:
@@ -155,11 +154,13 @@ cd GenAIExamples/CodeGen/docker_compose/intel/hpu/gaudi
155154
```
156155

157156
TGI service:
157+
158158
```bash
159159
docker compose --profile codegen-xeon-tgi up -d
160160
```
161161

162162
vLLM service:
163+
163164
```bash
164165
docker compose --profile codegen-xeon-vllm up -d
165166
```
@@ -175,16 +176,17 @@ cd GenAIExamples/CodeGen/docker_compose/intel/cpu/xeon
175176
```
176177

177178
TGI service:
179+
178180
```bash
179181
docker compose --profile codegen-xeon-tgi up -d
180182
```
181183

182184
vLLM service:
185+
183186
```bash
184187
docker compose --profile codegen-xeon-vllm up -d
185188
```
186189

187-
188190
Refer to the [Xeon Guide](./docker_compose/intel/cpu/xeon/README.md) for more instructions on building docker images from source.
189191

190192
### Deploy CodeGen on Kubernetes using Helm Chart

CodeGen/codegen.py

Lines changed: 26 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# Copyright (C) 2024 Intel Corporation
22
# SPDX-License-Identifier: Apache-2.0
33

4+
import ast
45
import asyncio
56
import os
6-
import ast
77

8-
from comps import MegaServiceEndpoint, MicroService, ServiceOrchestrator, ServiceRoleType, ServiceType, CustomLogger
8+
from comps import CustomLogger, MegaServiceEndpoint, MicroService, ServiceOrchestrator, ServiceRoleType, ServiceType
99
from comps.cores.mega.utils import handle_message
1010
from comps.cores.proto.api_protocol import (
1111
ChatCompletionRequest,
@@ -30,21 +30,21 @@
3030
TEI_EMBEDDING_HOST_IP = os.getenv("TEI_EMBEDDING_HOST_IP", "0.0.0.0")
3131
EMBEDDER_PORT = int(os.getenv("EMBEDDER_PORT", 6000))
3232

33-
grader_prompt = """You are a grader assessing relevance of a retrieved document to a user question. \n
33+
grader_prompt = """You are a grader assessing relevance of a retrieved document to a user question. \n
3434
Here is the user question: {question} \n
3535
Here is the retrieved document: \n\n {document} \n\n
3636
37-
If the document contains keywords related to the user question, grade it as relevant.
38-
It does not need to be a stringent test. The goal is to filter out erroneous retrievals.
37+
If the document contains keywords related to the user question, grade it as relevant.
38+
It does not need to be a stringent test. The goal is to filter out erroneous retrievals.
3939
Rules:
40-
- Do not return the question, the provided document or explanation.
41-
- if this document is relevant to the question, return 'yes' otherwise return 'no'.
40+
- Do not return the question, the provided document or explanation.
41+
- if this document is relevant to the question, return 'yes' otherwise return 'no'.
4242
- Do not include any other details in your response.
4343
"""
4444

45+
4546
def align_inputs(self, inputs, cur_node, runtime_graph, llm_parameters_dict, **kwargs):
46-
"""
47-
Aligns the inputs based on the service type of the current node.
47+
"""Aligns the inputs based on the service type of the current node.
4848
4949
Parameters:
5050
- self: Reference to the current instance of the class.
@@ -57,25 +57,21 @@ def align_inputs(self, inputs, cur_node, runtime_graph, llm_parameters_dict, **k
5757
Returns:
5858
- inputs: The aligned inputs for the current node.
5959
"""
60-
60+
6161
# Check if the current service type is EMBEDDING
6262
if self.services[cur_node].service_type == ServiceType.EMBEDDING:
6363
# Store the input query for later use
6464
self.input_query = inputs["query"]
6565
# Set the input for the embedding service
6666
inputs["input"] = inputs["query"]
67-
67+
6868
# Check if the current service type is RETRIEVER
6969
if self.services[cur_node].service_type == ServiceType.RETRIEVER:
7070
# Extract the embedding from the inputs
71-
embedding = inputs['data'][0]['embedding']
71+
embedding = inputs["data"][0]["embedding"]
7272
# Align the inputs for the retriever service
73-
inputs = {
74-
"index_name": llm_parameters_dict["index_name"],
75-
"text": self.input_query,
76-
"embedding": embedding
77-
}
78-
73+
inputs = {"index_name": llm_parameters_dict["index_name"], "text": self.input_query, "embedding": embedding}
74+
7975
return inputs
8076

8177

@@ -90,9 +86,7 @@ def __init__(self, host="0.0.0.0", port=8000):
9086
self.endpoint = str(MegaServiceEndpoint.CODE_GEN)
9187

9288
def add_remote_service(self):
93-
"""
94-
Adds remote microservices to the service orchestrators and defines the flow between them.
95-
"""
89+
"""Adds remote microservices to the service orchestrators and defines the flow between them."""
9690

9791
# Define the embedding microservice
9892
embedding = MicroService(
@@ -137,8 +131,7 @@ def add_remote_service(self):
137131
self.megaservice_llm.add(llm)
138132

139133
async def read_streaming_response(self, response: StreamingResponse):
140-
"""
141-
Reads the streaming response from a StreamingResponse object.
134+
"""Reads the streaming response from a StreamingResponse object.
142135
143136
Parameters:
144137
- self: Reference to the current instance of the class.
@@ -153,8 +146,7 @@ async def read_streaming_response(self, response: StreamingResponse):
153146
return body.decode("utf-8") # Decode the accumulated byte string to a regular string
154147

155148
async def handle_request(self, request: Request):
156-
"""
157-
Handles the incoming request, processes it through the appropriate microservices,
149+
"""Handles the incoming request, processes it through the appropriate microservices,
158150
and returns the response.
159151
160152
Parameters:
@@ -189,7 +181,7 @@ async def handle_request(self, request: Request):
189181
presence_penalty=chat_request.presence_penalty if chat_request.presence_penalty else 0.0,
190182
repetition_penalty=chat_request.repetition_penalty if chat_request.repetition_penalty else 1.03,
191183
stream=stream_opt,
192-
index_name=chat_request.index_name
184+
index_name=chat_request.index_name,
193185
)
194186

195187
# Initialize the initial inputs with the generated prompt
@@ -237,18 +229,20 @@ async def handle_request(self, request: Request):
237229
if r["choices"][0]["text"] == "yes":
238230
# Append the document to the relevant_docs list
239231
relevant_docs.append(doc)
240-
232+
241233
# Update the initial inputs with the relevant documents
242-
if len(relevant_docs)>0:
234+
if len(relevant_docs) > 0:
243235
logger.info(f"[ CodeGenService - handle_request ] {len(relevant_docs)} relevant document\s found.")
244236
query = initial_inputs["query"]
245237
initial_inputs = {}
246238
initial_inputs["retrieved_docs"] = relevant_docs
247239
initial_inputs["initial_query"] = query
248-
240+
249241
else:
250-
logger.info("[ CodeGenService - handle_request ] Could not find any relevant documents. The query will be used as input to the LLM.")
251-
242+
logger.info(
243+
"[ CodeGenService - handle_request ] Could not find any relevant documents. The query will be used as input to the LLM."
244+
)
245+
252246
else:
253247
# Use the combined retriever and LLM microservice
254248
megaservice = self.megaservice_retriever_llm
@@ -301,4 +295,4 @@ def start(self):
301295
if __name__ == "__main__":
302296
chatqna = CodeGenService(port=MEGA_SERVICE_PORT)
303297
chatqna.add_remote_service()
304-
chatqna.start()
298+
chatqna.start()

CodeGen/docker_compose/intel/cpu/xeon/README.md

Lines changed: 17 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,6 @@ Then run the command `docker images`, you will have the following Docker images:
183183
- `opea/codegen-ui:latest` (Optional)
184184
- `opea/codegen-react-ui:latest` (Optional)
185185

186-
187186
### Start the Docker Containers for All Services
188187

189188
CodeGen support TGI service and vLLM service, you can choose start either one of them.
@@ -229,16 +228,16 @@ docker compose --profile codegen-xeon-vllm up -d
229228

230229
3. Dataprep Microservice
231230

232-
Make sure to replace the file name placeholders with your correct file name
231+
Make sure to replace the file name placeholders with your correct file name
233232

234-
```bash
235-
curl http://${host_ip}:6007/v1/dataprep/ingest \
236-
-X POST \
237-
-H "Content-Type: multipart/form-data" \
238-
-F "files=@./file1.pdf" \
239-
-F "files=@./file2.txt" \
240-
-F "index_name=my_API_document"
241-
```
233+
```bash
234+
curl http://${host_ip}:6007/v1/dataprep/ingest \
235+
-X POST \
236+
-H "Content-Type: multipart/form-data" \
237+
-F "files=@./file1.pdf" \
238+
-F "files=@./file2.txt" \
239+
-F "index_name=my_API_document"
240+
```
242241

243242
4. MegaService
244243

@@ -248,16 +247,16 @@ docker compose --profile codegen-xeon-vllm up -d
248247
-d '{"messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
249248
```
250249

251-
CodeGen service with RAG and Agents activated based on an index.
252-
253-
```bash
254-
curl http://${host_ip}:7778/v1/codegen \
255-
-H "Content-Type: application/json" \
256-
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
257-
```
258-
250+
CodeGen service with RAG and Agents activated based on an index.
251+
252+
```bash
253+
curl http://${host_ip}:7778/v1/codegen \
254+
-H "Content-Type: application/json" \
255+
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
256+
```
259257

260258
## 🚀 Launch the Gradio Based UI (Recommended)
259+
261260
To access the Gradio frontend URL, follow the steps in [this README](../../../../ui/gradio/README.md)
262261

263262
Code Generation Tab
@@ -274,7 +273,6 @@ Here is an example of running a query in the Gradio UI using an Index:
274273

275274
![project-screenshot](../../../../assets/img/codegen_gradio_ui_query.png)
276275

277-
278276
## 🚀 Launch the Svelte Based UI (Optional)
279277

280278
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`. By default, the UI runs on port 5173 internally. If you prefer to use a different host port to access the frontend, you can modify the port mapping in the `compose.yaml` file as shown below:

CodeGen/docker_compose/intel/cpu/xeon/compose.yaml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# Copyright (C) 2025 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
services:
25

36
tgi-service:
@@ -132,7 +135,7 @@ services:
132135
http_proxy: ${http_proxy}
133136
https_proxy: ${https_proxy}
134137
REDIS_URL: ${REDIS_URL}
135-
REDIS_HOST: ${host_ip}
138+
REDIS_HOST: ${host_ip}
136139
INDEX_NAME: ${INDEX_NAME}
137140
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
138141
LOGFLAG: true
@@ -197,4 +200,4 @@ services:
197200
restart: unless-stopped
198201
networks:
199202
default:
200-
driver: bridge
203+
driver: bridge

CodeGen/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 22 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,7 @@ Then run the command `docker images`, you will have the following Docker images:
177177
- `opea/codegen-react-ui:latest` (Optional)
178178

179179
### Start the Docker Containers for All Services
180+
180181
#### Deploy CodeGen on Gaudi
181182

182183
Find the corresponding [compose.yaml](./docker_compose/intel/hpu/gaudi/compose.yaml). User could start CodeGen based on TGI or vLLM service:
@@ -186,11 +187,13 @@ cd GenAIExamples/CodeGen/docker_compose/intel/hpu/gaudi
186187
```
187188

188189
TGI service:
190+
189191
```bash
190192
docker compose --profile codegen-gaudi-tgi up -d
191193
```
192194

193195
vLLM service:
196+
194197
```bash
195198
docker compose --profile codegen-gaudi-vllm up -d
196199
```
@@ -206,11 +209,13 @@ cd GenAIExamples/CodeGen/docker_compose/intel/cpu/xeon
206209
```
207210

208211
TGI service:
212+
209213
```bash
210214
docker compose --profile codegen-xeon-tgi up -d
211215
```
212216

213217
vLLM service:
218+
214219
```bash
215220
docker compose --profile codegen-xeon-vllm up -d
216221
```
@@ -238,16 +243,16 @@ docker compose --profile codegen-xeon-vllm up -d
238243

239244
3. Dataprep Microservice
240245

241-
Make sure to replace the file name placeholders with your correct file name
246+
Make sure to replace the file name placeholders with your correct file name
242247

243-
```bash
244-
curl http://${host_ip}:6007/v1/dataprep/ingest \
245-
-X POST \
246-
-H "Content-Type: multipart/form-data" \
247-
-F "files=@./file1.pdf" \
248-
-F "files=@./file2.txt" \
249-
-F "index_name=my_API_document"
250-
```
248+
```bash
249+
curl http://${host_ip}:6007/v1/dataprep/ingest \
250+
-X POST \
251+
-H "Content-Type: multipart/form-data" \
252+
-F "files=@./file1.pdf" \
253+
-F "files=@./file2.txt" \
254+
-F "index_name=my_API_document"
255+
```
251256

252257
4. MegaService
253258

@@ -257,15 +262,16 @@ docker compose --profile codegen-xeon-vllm up -d
257262
-d '{"messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
258263
```
259264

260-
CodeGen service with RAG and Agents activated based on an index.
261-
262-
```bash
263-
curl http://${host_ip}$:7778/v1/codegen \
264-
-H "Content-Type: application/json" \
265-
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
266-
```
265+
CodeGen service with RAG and Agents activated based on an index.
266+
267+
```bash
268+
curl http://${host_ip}$:7778/v1/codegen \
269+
-H "Content-Type: application/json" \
270+
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
271+
```
267272

268273
## 🚀 Launch the Gradio Based UI (Recommended)
274+
269275
To access the Gradio frontend URL, follow the steps in [this README](../../../../ui/gradio/README.md)
270276

271277
Code Generation Tab
@@ -282,7 +288,6 @@ Here is an example of running a query in the Gradio UI using an Index:
282288

283289
![project-screenshot](../../../../assets/img/codegen_gradio_ui_query.png)
284290

285-
286291
## 🚀 Launch the Svelte Based UI (Optional)
287292

288293
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`. By default, the UI runs on port 5173 internally. If you prefer to use a different host port to access the frontend, you can modify the port mapping in the `compose.yaml` file as shown below:
@@ -392,4 +397,3 @@ For example:
392397
- Ask question and get answer
393398

394399
![qna](../../../../assets/img/codegen_qna.png)
395-

CodeGen/docker_compose/intel/hpu/gaudi/compose.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ services:
151151
http_proxy: ${http_proxy}
152152
https_proxy: ${https_proxy}
153153
REDIS_URL: ${REDIS_URL}
154-
REDIS_HOST: ${host_ip}
154+
REDIS_HOST: ${host_ip}
155155
INDEX_NAME: ${INDEX_NAME}
156156
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
157157
LOGFLAG: true
@@ -216,4 +216,4 @@ services:
216216
restart: unless-stopped
217217
networks:
218218
default:
219-
driver: bridge
219+
driver: bridge

0 commit comments

Comments
 (0)