Howt to convert llama2 7B.pth file into ggml-model.bin

[Alkahwaji]

I downloaded Llama2 7B files (consolidated.00.pth - checklist.chk - tokenizer.model). I want to load this model using llama-cpp but first, i need to convert this model into a bin file. What should I do?

[harshal09]

You might want to quatized as well follow below link and let me know if it helps.
https://medium.com/towards-data-science/4-bit-quantization-with-gptq-36b0f4f02c34

[rai-shi]

Hi, did you find any way to convert model into a bin file? Because the link which is shared is not helped me.

[bucketsize]

After downloading the .pth (pytorch) file (assuming you downloaded from meta), you need to convert it to the Q4_0 quantized ggml model.

• convert from .pth -> hugginface format
  python -m transformers.models.llama.convert_llama_weights_to_hf --model_size 7B --input_dir llama-2-7b-chat/ --output_dir llama-2-7b-chat-hf/
• convert from huggingface to ggml F16 format
  cd llama.cpp/ python3 -m pip install -r requirements.txt mkdir models/7B python3 convert.py ../llama/llama-2-7b-chat-hf/ --outfile models/7B/ggml-model-f16.bin --outtype f16 --vocab-dir ../llama/llama-2-7b-chat-hf/
• quantize to Q4_0 ggml format (reduces size and precision of model weights)
  ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0 ./main -m ./models/7B/ggml-model-q4_0.bin -n 1024 --color -i -r "User:"

[hhy-huang]

I solved my issues with your solution, thank you.

[hasanradi93]

How you solved?
I download llama-2-13-chat from meta
and o download llama-cpp
i need to know how to convert the consolidated.00.pth and how i use (run it)it in a script?
can you help me?