Skip to content

Commit 0e48174

Browse files
heheda12345ywang96
authored andcommitted
[Bugfix] use blockmanagerv1 for encoder-decoder (vllm-project#9084)
Co-authored-by: Roger Wang <ywang@roblox.com> Signed-off-by: Sumit Dubey <sumit.dubey2@ibm.com>
1 parent 1e83ddf commit 0e48174

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

vllm/engine/arg_utils.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -903,6 +903,11 @@ def create_engine_config(self) -> EngineConfig:
903903
"--enable-prefix-caching is currently not "
904904
"supported for multimodal models and has been disabled.")
905905
self.enable_prefix_caching = False
906+
if model_config.is_encoder_decoder_model:
907+
logger.warning(
908+
"Block Manager v2 does not support encoder-decoder models"
909+
" currently. Using Block Manager v1 as fallback.")
910+
self.use_v2_block_manager = False
906911

907912
cache_config = CacheConfig(
908913
block_size=self.block_size if self.device != "neuron" else

0 commit comments

Comments
 (0)