Help with illegal hardware instruction

[SpinNeutron]

Hello all!

While running the example high level api script,

from llama_cpp import Llama
llm = Llama(model_path="model_in_this_folder.gguf")
...

No matter how I define model_path I get the error zsh: illegal hardware instruction python3 test.py or just Illegal instruction from bash.

I installed using pip install llama-cpp-python
I tried reinstalling with pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir to no avail.

The code causing the error is llm = Llama(anything).

I assume that my installation has compiled for different hardware than what I'm running on. Any ideas on how to fix this?

[JnJarvis]

I also have this problem.

[JnJarvis]

I may have found a solution if you're still looking for one.

I tinkered around with the solutions suggested in #272
The CMAKE_ARGS that worked for me were:
CMAKE_ARGS="-DLLAMA_NATIVE=OFF -DGGML_CUDA=ON -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF -DFORCE_CMAKE=ON"
I'm running it on a older CPU, that doesn't support AVX2, and I expect that is your case.
set them and run:
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121 --upgrade --force-reinstall --no-cache-dir

Hope this helps!

[SpinNeutron]

Thanks! I was using a CPU without AVX2. At some point I gave up completely on getting it to work. I used the python subprocess library to call llamacpp from my programs.

Huggingface supports gguf and llamacpp has so much built into now, I never looked back.

Anyway, thanks again!