You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when I try this
vllm serve neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 --host 0.0.0.0 --port 8000 --tensor-parallel-size 8 --seed 1234 --enable_prefix_caching --enable-chunked-prefill --max-model-len 32000 --num-scheduler-steps 8
I got this
raise ValueError("Chunked prefill is not supported with "
ValueError: Chunked prefill is not supported with multi-step (--num-scheduler-steps > 1)
ERROR 09-08 13:11:00 api_server.py:186] RPCServer process died before responding to readiness probe
is it an expected behavior?
Before submitting a new issue...
Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
The text was updated successfully, but these errors were encountered:
This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!
Your current environment
vllm 0.6.0
🐛 Describe the bug
when I try this
vllm serve neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 --host 0.0.0.0 --port 8000 --tensor-parallel-size 8 --seed 1234 --enable_prefix_caching --enable-chunked-prefill --max-model-len 32000 --num-scheduler-steps 8
I got this
raise ValueError("Chunked prefill is not supported with "
ValueError: Chunked prefill is not supported with multi-step (--num-scheduler-steps > 1)
ERROR 09-08 13:11:00 api_server.py:186] RPCServer process died before responding to readiness probe
is it an expected behavior?
Before submitting a new issue...
The text was updated successfully, but these errors were encountered: