Skip to content

Commit 26478ab

Browse files
committed
docs: Update README.md
1 parent b14dd98 commit 26478ab

File tree

1 file changed

+10
-6
lines changed

1 file changed

+10
-6
lines changed

README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -529,19 +529,22 @@ Then you'll need to use a custom chat handler to load the clip model and process
529529
You can also pull the model from the Hugging Face Hub using the `from_pretrained` method.
530530

531531
```python
532-
>>> from llama_cpp import Llama
533-
>>> from llama_cpp.llama_chat_format import MoondreamChatHandler
534-
>>> chat_handler = MoondreamChatHandler.from_pretrained(
532+
from llama_cpp import Llama
533+
from llama_cpp.llama_chat_format import MoondreamChatHandler
534+
535+
chat_handler = MoondreamChatHandler.from_pretrained(
535536
repo_id="vikhyatk/moondream2",
536537
filename="*mmproj*",
537538
)
538-
>>> llm = Llama.from_pretrained(
539-
repo_id="vikhyatk/moondream2"
539+
540+
llm = Llama.from_pretrained(
541+
repo_id="vikhyatk/moondream2",
540542
filename="*text-model*",
541543
chat_handler=chat_handler,
542544
n_ctx=2048, # n_ctx should be increased to accomodate the image embedding
543545
)
544-
>>> llm.create_chat_completion(
546+
547+
respoonse = llm.create_chat_completion(
545548
messages = [
546549
{
547550
"role": "user",
@@ -553,6 +556,7 @@ You can also pull the model from the Hugging Face Hub using the `from_pretrained
553556
}
554557
]
555558
)
559+
print(response["choices"][0]["text"])
556560
```
557561

558562
**Note**: Multi-modal models also support tool calling and JSON mode.

0 commit comments

Comments
 (0)