Skip to content

Commit 631c562

Browse files
comaniacgarg-amit
authored andcommitted
[MISC] Skip dumping inputs when unpicklable (vllm-project#8744)
Signed-off-by: Amit Garg <mitgarg17495@gmail.com>
1 parent 3a347ac commit 631c562

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

vllm/worker/model_runner_base.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,15 @@ def _wrapper(*args, **kwargs):
137137
for t in kv_caches
138138
if is_tensor(t)]
139139

140-
pickle.dump(dumped_inputs, filep)
140+
try:
141+
pickle.dump(dumped_inputs, filep)
142+
except Exception as pickle_err:
143+
logger.warning(
144+
"Failed to pickle inputs of failed execution: %s",
145+
str(pickle_err))
146+
raise type(err)(f"Error in model execution: "
147+
f"{str(err)}") from err
148+
141149
logger.info(
142150
"Completed writing input of failed execution to %s.",
143151
filename)

0 commit comments

Comments
 (0)