Replies: 1 comment
-
I don't use that model, however this usually happens due to limitations of the model itself. Also, quantization can impair code generation or so I have heard. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
llama.cpp version: llama-b4938-bin-win-cuda-cu12.4-x64.zip
llm: bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf
llama-server.exe -m ..\models\mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf -ngl 99 --temp 0.6 -fa -c 20480 --metrics
I ask it to rewrite a python script,
the script is looking fine,
but it insert a lot of random empty space in the python script which make it not working,
Do anyone have similiar issue or just mine?
Beta Was this translation helpful? Give feedback.
All reactions