Skip to content

Commit ac55d0a

Browse files
committed
fix: Clear kv cache to avoid kv bug when image is evaluated first
1 parent 4badac3 commit ac55d0a

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

llama_cpp/llama_chat_format.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2637,6 +2637,7 @@ def embed_image_bytes(image_bytes: bytes):
26372637

26382638
# Evaluate prompt
26392639
llama.reset()
2640+
llama._ctx.kv_cache_clear()
26402641
for type_, value in split_text:
26412642
if type_ == "text":
26422643
tokens = llama.tokenize(value.encode("utf8"), add_bos=False, special=True)

0 commit comments

Comments
 (0)