Skip to content

Commit 4cc120c

Browse files
authored
infill : add download instructions for model (ggml-org#6626)
* infill : add download instructions for model This commit adds instructions on how to download a CodeLlama model using the `hf.sh` script. This will download the model and place it in the `models` directory which is the same model use later by the infill example. Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com> * squash! infill : add download instructions for model Clarify the reason for using CodeLlama. Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com> --------- Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
1 parent 24ee66e commit 4cc120c

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

examples/infill/README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ The `infill` program offers a seamless way to interact with LLaMA models, allowi
3636

3737
### Example
3838

39+
Download a model that supports infill, for example CodeLlama:
40+
```console
41+
scripts/hf.sh --repo TheBloke/CodeLlama-13B-GGUF --file codellama-13b.Q5_K_S.gguf --outdir models
42+
```
43+
3944
```bash
4045
./infill -t 10 -ngl 0 -m models/codellama-13b.Q5_K_S.gguf -c 4096 --temp 0.7 --repeat_penalty 1.1 -n 20 --in-prefix "def helloworld():\n print(\"hell" --in-suffix "\n print(\"goodbye world\")\n "
4146
```

0 commit comments

Comments
 (0)