Skip to content

Commit 082a9fe

Browse files
committed
Fix Q4 cache in chat example
1 parent 0b05686 commit 082a9fe

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

examples/chat.py

+2
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,8 @@
132132

133133
if args.cache_8bit:
134134
cache = ExLlamaV2Cache_8bit(model, lazy = not model.loaded)
135+
elif args.cache_q4:
136+
cache = ExLlamaV2Cache_Q4(model, lazy = not model.loaded)
135137
else:
136138
cache = ExLlamaV2Cache(model, lazy = not model.loaded)
137139

0 commit comments

Comments
 (0)