Skip to content

Commit b6f0e80

Browse files
Update text-generation.md (mudler#2095)
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>
1 parent 729378c commit b6f0e80

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

docs/content/docs/features/text-generation.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,10 @@ parameters:
257257
# swap_space: 2
258258
# Uncomment to specify the maximum length of a sequence (including prompt and output)
259259
# max_model_len: 32768
260+
# Uncomment and specify the number of Tensor divisions.
261+
# Allows you to partition and run large models. Performance gains are limited.
262+
# https://github.com/vllm-project/vllm/issues/1435
263+
# tensor_parallel_size: 2
260264
```
261265

262266
The backend will automatically download the required files in order to run the model.
@@ -356,4 +360,4 @@ template:
356360
357361
completion: |
358362
{{.Input}}
359-
```
363+
```

0 commit comments

Comments
 (0)