Skip to content

Commit bf5c391

Browse files
JoshuaL3000pre-commit-ci[bot]
JoshuaL3000
and
pre-commit-ci[bot]
authored
Add Workflow Executor Example (#892)
Signed-off-by: JoshuaL3000 <joshua.jian.ern.liew@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent c65d7d4 commit bf5c391

File tree

16 files changed

+731
-0
lines changed

16 files changed

+731
-0
lines changed

‎WorkflowExecAgent/README.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Workflow Executor Agent
2+
3+
## Overview
4+
5+
GenAI Workflow Executor Example showcases the capability to handle data/AI workflow operations via LangChain agents to execute custom-defined workflow-based tools. These workflow tools can be interfaced from any 3rd-party tools in the market (no-code/low-code/IDE) such as Alteryx, RapidMiner, Power BI, Intel Data Insight Automation which allows users to create complex data/AI workflow operations for different use-cases.
6+
7+
### Workflow Executor
8+
9+
This example demonstrates a single React-LangGraph with a `Workflow Executor` tool to ingest a user prompt to execute workflows and return an agent reasoning response based on the workflow output data.
10+
11+
First the LLM extracts the relevant information from the user query based on the schema of the tool in `tools/tools.yaml`. Then the agent sends this `AgentState` to the `Workflow Executor` tool.
12+
13+
`Workflow Executor` tool uses `EasyDataSDK` class as seen under `tools/sdk.py` to interface with several high-level API's. There are 3 steps to this tool implementation:
14+
15+
1. Starts the workflow with workflow parameters and workflow id extracted from the user query.
16+
17+
2. Periodically checks the workflow status for completion or failure. This may be through a database which stores the current status of the workflow
18+
19+
3. Retrieves the output data from the workflow through a storage service.
20+
21+
The `AgentState` is sent back to the LLM for reasoning. Based on the output data, the LLM generates a response to answer the user's input prompt.
22+
23+
Below shows an illustration of this flow:
24+
25+
![image](https://github.com/user-attachments/assets/cb135042-1505-4aef-8822-c78c2f72aa2a)
26+
27+
### Workflow Serving for Agent
28+
29+
As an example, here we have a Churn Prediction use-case workflow as the serving workflow for the agent execution. It is created through Intel Data Insight Automation platform. The image below shows a snapshot of the Churn Prediction workflow.
30+
31+
![image](https://github.com/user-attachments/assets/c067f8b3-86cf-4abc-a8bd-51a98de8172d)
32+
33+
The workflow contains 2 paths which can be seen in the workflow illustrated, the top path and bottom path.
34+
35+
1. Top path - The training path which ends at the random forest classifier node is the training path. The data is cleaned through a series of nodes and used to train a random forest model for prediction.
36+
37+
2. Bottom path - The inference path where trained random forest model is used for inferencing based on input parameter.
38+
39+
For this agent workflow execution, the inferencing path is executed to yield the final output result of the `Model Predictor` node. The same output is returned to the `Workflow Executor` tool through the `Langchain API Serving` node.
40+
41+
There are `Serving Parameters` in the workflow, which are the tool input variables used to start a workflow instance obtained from `params` the LLM extracts from the user query. Below shows the parameter configuration option for the Intel Data Insight Automation workflow UI.
42+
43+
![image](https://github.com/user-attachments/assets/ce8ef01a-56ff-4278-b84d-b6e4592b28c6)
44+
45+
Manually running the workflow yields the tabular data output as shown below:
46+
47+
![image](https://github.com/user-attachments/assets/241c1aba-2a24-48da-8005-ec7bfe657179)
48+
49+
In the workflow serving for agent, this output will be returned to the `Workflow Executor` tool. The LLM can then answer the user's original question based on this output.
50+
51+
To start prompting the agent microservice, we will use the following command for this use case:
52+
53+
```sh
54+
curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
55+
"query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
56+
}'
57+
```
58+
59+
The user has to provide a `workflow_id` and workflow `params` in the query. `workflow_id` a unique id used for serving the workflow to the microservice. Notice that the `query` string includes all the workflow `params` which the user has defined in the workflow. The LLM will extract these parameters into a dictionary format for the workflow `Serving Parameters` as shown below:
60+
61+
```python
62+
params = {"gender": "Female", "tenure": 55, "MonthlyAvgCharges": 103.7}
63+
```
64+
65+
These parameters will be passed into the `Workflow Executor` tool to start the workflow execution of specified `workflow_id`. Thus, everything will be handled via the microservice.
66+
67+
And finally here are the results from the microservice logs:
68+
69+
![image](https://github.com/user-attachments/assets/969fefb7-543d-427f-a56c-dc70e474ae60)
70+
71+
## Microservice Setup
72+
73+
### Start Agent Microservice
74+
75+
Workflow Executor will have a single docker image. First, build the agent docker image.
76+
77+
```sh
78+
git clone https://github.com/opea-project/GenAIExamples.git
79+
cd GenAIExamples//WorkflowExecAgent/docker_image_build/
80+
docker compose -f build.yaml build --no-cache
81+
```
82+
83+
Configure `GenAIExamples/WorkflowExecAgent/docker_compose/.env` file with the following. Replace the variables according to your usecase.
84+
85+
```sh
86+
export SDK_BASE_URL=${SDK_BASE_URL}
87+
export SERVING_TOKEN=${SERVING_TOKEN}
88+
export HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}
89+
export llm_engine=${llm_engine}
90+
export llm_endpoint_url=${llm_endpoint_url}
91+
export ip_address=$(hostname -I | awk '{print $1}')
92+
export model="mistralai/Mistral-7B-Instruct-v0.3"
93+
export recursion_limit=${recursion_limit}
94+
export temperature=0
95+
export max_new_tokens=1000
96+
export WORKDIR=${WORKDIR}
97+
export TOOLSET_PATH=$WORKDIR/GenAIExamples/WorkflowExecAgent/tools/
98+
export http_proxy=${http_proxy}
99+
export https_proxy=${https_proxy}
100+
```
101+
102+
Launch service by running the docker compose command.
103+
104+
```sh
105+
cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_compose
106+
docker compose -f compose.yaml up -d
107+
```
108+
109+
### Validate service
110+
111+
The microservice logs can be viewed using:
112+
113+
```sh
114+
docker logs workflowexec-agent-endpoint
115+
```
116+
117+
You should be able to see "HTTP server setup successful" upon successful startup.
118+
119+
You can validate the service using the following command:
120+
121+
```sh
122+
curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
123+
"query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
124+
}'
125+
```
126+
127+
Update the `query` with the workflow parameters, workflow id, etc based on the workflow context.
128+
129+
## Roadmap
130+
131+
Phase II: Agent memory integration to enable capability to store tool intermediate results, such as workflow instance key.

‎WorkflowExecAgent/docker_compose/intel/cpu/xeon/compose_vllm.yaml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
services:
5+
worflowexec-agent:
6+
image: opea/agent-langchain:latest
7+
container_name: workflowexec-agent-endpoint
8+
volumes:
9+
- ${WORKDIR}/GenAIComps/comps/agent/langchain/:/home/user/comps/agent/langchain/
10+
- ${TOOLSET_PATH}:/home/user/tools/
11+
ports:
12+
- "9090:9090"
13+
ipc: host
14+
environment:
15+
ip_address: ${ip_address}
16+
strategy: react_langgraph
17+
recursion_limit: ${recursion_limit}
18+
llm_engine: ${llm_engine}
19+
llm_endpoint_url: ${llm_endpoint_url}
20+
model: ${model}
21+
temperature: ${temperature}
22+
max_new_tokens: ${max_new_tokens}
23+
streaming: false
24+
tools: /home/user/tools/tools.yaml
25+
no_proxy: ${no_proxy}
26+
http_proxy: ${http_proxy}
27+
https_proxy: ${https_proxy}
28+
port: 9090
29+
SDK_BASE_URL: ${SDK_BASE_URL}
30+
SERVING_TOKEN: ${SERVING_TOKEN}
31+
custom_prompt: /home/user/tools/custom_prompt.py

‎WorkflowExecAgent/docker_image_build/build.yaml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
services:
5+
agent-langchain:
6+
build:
7+
context: GenAIComps
8+
dockerfile: comps/agent/langchain/Dockerfile
9+
args:
10+
http_proxy: ${http_proxy}
11+
https_proxy: ${https_proxy}
12+
no_proxy: ${no_proxy}
13+
image: ${REGISTRY:-opea}/agent-langchain:${TAG:-latest}

‎WorkflowExecAgent/tests/1_build_images.sh

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/bin/bash
2+
# Copyright (C) 2024 Intel Corporation
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
set -e
6+
WORKPATH=$(dirname "$PWD")
7+
export WORKDIR=$WORKPATH/../../
8+
echo "WORKDIR=${WORKDIR}"
9+
10+
function get_genai_comps() {
11+
if [ ! -d "GenAIComps" ] ; then
12+
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
13+
fi
14+
}
15+
16+
function build_agent_docker_image() {
17+
cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_image_build/
18+
get_genai_comps
19+
echo "Build agent image with --no-cache..."
20+
docker compose -f build.yaml build --no-cache
21+
}
22+
23+
function main() {
24+
echo "==================== Build agent docker image ===================="
25+
build_agent_docker_image
26+
echo "==================== Build agent docker image completed ===================="
27+
}
28+
29+
main

‎WorkflowExecAgent/tests/2_start_vllm_service.sh

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/bin/bash
2+
# Copyright (C) 2024 Intel Corporation
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
set -e
6+
7+
WORKPATH=$(dirname "$PWD")
8+
LOG_PATH="$WORKPATH/tests"
9+
vllm_port=${vllm_port}
10+
[[ -z "$vllm_port" ]] && vllm_port=8084
11+
model=mistralai/Mistral-7B-Instruct-v0.3
12+
export WORKDIR=$WORKPATH/../../
13+
export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
14+
15+
function build_vllm_docker_image() {
16+
echo "Building the vllm docker images"
17+
cd $WORKPATH
18+
echo $WORKPATH
19+
if [ ! -d "./vllm" ]; then
20+
git clone https://github.com/vllm-project/vllm.git
21+
cd ./vllm; git checkout tags/v0.6.0
22+
else
23+
cd ./vllm
24+
fi
25+
docker build -f Dockerfile.cpu -t vllm-cpu-env --shm-size=100g .
26+
if [ $? -ne 0 ]; then
27+
echo "opea/vllm:cpu failed"
28+
exit 1
29+
else
30+
echo "opea/vllm:cpu successful"
31+
fi
32+
}
33+
34+
function start_vllm_service() {
35+
echo "start vllm service"
36+
docker run -d -p ${vllm_port}:${vllm_port} --rm --network=host --name test-comps-vllm-service -v ~/.cache/huggingface:/root/.cache/huggingface -v ${WORKPATH}/tests/tool_chat_template_mistral_custom.jinja:/root/tool_chat_template_mistral_custom.jinja -e HF_TOKEN=$HF_TOKEN -e http_proxy=$http_proxy -e https_proxy=$https_proxy -it vllm-cpu-env --model ${model} --port ${vllm_port} --chat-template /root/tool_chat_template_mistral_custom.jinja --enable-auto-tool-choice --tool-call-parser mistral
37+
echo ${LOG_PATH}/vllm-service.log
38+
sleep 5s
39+
echo "Waiting vllm ready"
40+
n=0
41+
until [[ "$n" -ge 100 ]] || [[ $ready == true ]]; do
42+
docker logs test-comps-vllm-service &> ${LOG_PATH}/vllm-service.log
43+
n=$((n+1))
44+
if grep -q "Uvicorn running on" ${LOG_PATH}/vllm-service.log; then
45+
break
46+
fi
47+
if grep -q "No such container" ${LOG_PATH}/vllm-service.log; then
48+
echo "container test-comps-vllm-service not found"
49+
exit 1
50+
fi
51+
sleep 5s
52+
done
53+
sleep 5s
54+
echo "Service started successfully"
55+
}
56+
57+
function main() {
58+
echo "==================== Build vllm docker image ===================="
59+
build_vllm_docker_image
60+
echo "==================== Build vllm docker image completed ===================="
61+
62+
echo "==================== Start vllm docker service ===================="
63+
start_vllm_service
64+
echo "==================== Start vllm docker service completed ===================="
65+
}
66+
67+
main

‎WorkflowExecAgent/tests/3_launch_and_validate_agent.sh

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
#!/bin/bash
2+
# Copyright (C) 2024 Intel Corporation
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
set -e
6+
7+
WORKPATH=$(dirname "$PWD")
8+
workflow_id=9809
9+
vllm_port=${vllm_port}
10+
[[ -z "$vllm_port" ]] && vllm_port=8084
11+
export WORKDIR=$WORKPATH/../../
12+
echo "WORKDIR=${WORKDIR}"
13+
export SDK_BASE_URL=${SDK_BASE_URL}
14+
export SERVING_TOKEN=${SERVING_TOKEN}
15+
export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
16+
export llm_engine=vllm
17+
export ip_address=$(hostname -I | awk '{print $1}')
18+
export llm_endpoint_url=http://${ip_address}:${vllm_port}
19+
export model=mistralai/Mistral-7B-Instruct-v0.3
20+
export recursion_limit=25
21+
export temperature=0
22+
export max_new_tokens=1000
23+
export TOOLSET_PATH=$WORKDIR/GenAIExamples/WorkflowExecAgent/tools/
24+
25+
function start_agent_and_api_server() {
26+
echo "Starting Agent services"
27+
cd $WORKDIR/GenAIExamples/WorkflowExecAgent/docker_compose/intel/cpu/xeon
28+
WORKDIR=$WORKPATH/docker_image_build/ docker compose -f compose_vllm.yaml up -d
29+
echo "Waiting agent service ready"
30+
sleep 5s
31+
}
32+
33+
function validate() {
34+
local CONTENT="$1"
35+
local EXPECTED_RESULT="$2"
36+
local SERVICE_NAME="$3"
37+
38+
if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then
39+
echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT"
40+
echo "[TEST INFO]: Workflow Executor agent service PASSED"
41+
else
42+
echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT"
43+
echo "[TEST INFO]: Workflow Executor agent service FAILED"
44+
fi
45+
}
46+
47+
function validate_agent_service() {
48+
echo "----------------Test agent ----------------"
49+
local CONTENT=$(curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
50+
"query": "I have a data with gender Female, tenure 55, MonthlyAvgCharges 103.7. Predict if this entry will churn. My workflow id is '${workflow_id}'."
51+
}')
52+
validate "$CONTENT" "The entry is not likely to churn" "workflowexec-agent-endpoint"
53+
docker logs workflowexec-agent-endpoint
54+
}
55+
56+
function main() {
57+
echo "==================== Start agent ===================="
58+
start_agent_and_api_server
59+
echo "==================== Agent started ===================="
60+
61+
echo "==================== Validate agent service ===================="
62+
validate_agent_service
63+
echo "==================== Agent service validated ===================="
64+
}
65+
66+
main

‎WorkflowExecAgent/tests/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Validate Workflow Agent Microservice
2+
3+
Microservice validation for Intel Data Insight Automation platform workflow serving.
4+
5+
## Usage
6+
7+
Configure necessary variables as listed below. Replace the variables according to your usecase.
8+
9+
```sh
10+
export SDK_BASE_URL=${SDK_BASE_URL}
11+
export SERVING_TOKEN=${SERVING_TOKEN}
12+
export HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}
13+
export workflow_id=${workflow_id} # workflow_id of the serving workflow
14+
export vllm_port=${vllm_port} # vllm serving port
15+
export ip_address=$(hostname -I | awk '{print $1}')
16+
export VLLM_CPU_OMP_THREADS_BIND=${VLLM_CPU_OMP_THREADS_BIND}
17+
export http_proxy=${http_proxy}
18+
export https_proxy=${https_proxy}
19+
```
20+
21+
Note: `SDK_BASE_URL` and `SERVING_TOKEN` can be obtained from Intel Data Insight Automation platform.
22+
23+
Launch validation by running the following command.
24+
25+
```sh
26+
cd GenAIExamples/WorkflowExecAgent/tests
27+
. /test_compose_on_xeon.sh
28+
```
29+
30+
`test_compose_on_xeon.sh` will run the other `.sh` files under `tests/`. The validation script launches 1 docker container for the agent microservice, and another for the vllm model serving on CPU. When validation is completed, all containers will be stopped.
31+
32+
The validation is tested by checking if the model reasoning output response matches a partial substring. The expected output is shown below:
33+
34+
![image](https://github.com/user-attachments/assets/88081bc8-7b73-470d-970e-92e0fe5f96ec)
35+
36+
## Note
37+
38+
- Currently the validation test is only designed with vllm model serving (CPU only).

0 commit comments

Comments
 (0)