Skip to content

Commit 1a934af

Browse files
ctao456srinarayan-srikanthanpre-commit-ci[bot]letonghanXinyuYe-Intel
authored
Add table to list port, endpoint, framework, model, serving, and hardware for each microservice in ChatQnA (#697)
Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com> Signed-off-by: Chun Tao <chun.tao@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Yue, Wenjiao <wenjiao.yue@intel.com> Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> Co-authored-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Zhenzhong1 <109137058+Zhenzhong1@users.noreply.github.com> Co-authored-by: WenjiaoYue <wenjiao.yue@intel.com> Co-authored-by: Lianhao Lu <lianhao.lu@intel.com> Co-authored-by: Ying Hu <ying.hu@intel.com>
1 parent 5c67204 commit 1a934af

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

ChatQnA/README.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,21 @@ flowchart LR
9797

9898
This ChatQnA use case performs RAG using LangChain, Redis VectorDB and Text Generation Inference on Intel Gaudi2 or Intel XEON Scalable Processors. The Intel Gaudi2 accelerator supports both training and inference for deep learning models in particular for LLMs. Visit [Habana AI products](https://habana.ai/products) for more details.
9999

100+
In the below, we provide a table that describes for each microservice component in the ChatQnA architecture, the default configuration of the open source project, hardware, port, and endpoint.
101+
102+
<details>
103+
<summary><b>Gaudi default compose.yaml</b></summary>
104+
105+
| MicroService | Open Source Project | HW | Port | Endpoint |
106+
| ------------ | ------------------- | ----- | ---- | -------------------- |
107+
| Embedding | Langchain | Xeon | 6000 | /v1/embaddings |
108+
| Retriever | Langchain, Redis | Xeon | 7000 | /v1/retrieval |
109+
| Reranking | Langchain, TEI | Gaudi | 8000 | /v1/reranking |
110+
| LLM | Langchain, TGI | Gaudi | 9000 | /v1/chat/completions |
111+
| Dataprep | Redis, Langchain | Xeon | 6007 | /v1/dataprep |
112+
113+
</details>
114+
100115
## Deploy ChatQnA Service
101116

102117
The ChatQnA service can be effortlessly deployed on either Intel Gaudi2 or Intel XEON Scalable Processors.

0 commit comments

Comments
 (0)