Skip to content

Commit d9a76f7

Browse files
Add further details for Kserve support in LMEval (#51)
1 parent 12122d8 commit d9a76f7

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

docs/modules/ROOT/pages/lm-eval-tutorial.adoc

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -685,6 +685,11 @@ In the case where both managed and existing PVCs are referenced in `outputs`, th
685685

686686
=== Using an `InferenceService`
687687

688+
[NOTE]
689+
====
690+
When using Kserve `InferenceServices`, only HuggingFace- and VLLM-based model serving runtimes are supported with the following method. Additionally, ensure DNS is set up for your cluster, as there is currently no method to pass additional request headers to a served model in `lm-evaluation-harness`.
691+
====
692+
688693
[NOTE]
689694
====
690695
This example assumes vLLM model already deployed in your cluster.

0 commit comments

Comments
 (0)