In chat mode, the LLM agent seems to keep talking to itself without stopping.

[Marvin-BW]

~/distributed-llama$ ./dllama chat --model ./converter/dllama_model_mlabonne--Meta-Llama-3.1-8B-Instruct-ablitera
ted_q40.m --tokenizer ./converter/dllama_tokenizer_mlabonne--Meta-Llama-3.1-8B-Instruct-abliterated.t --buffer-float-type q80 --steps 16 --nthreads 4 --workers 10.0.3.184:52415 10.0.1.200:52415 10.0.3.186:52415
📄 BosId: 128000 (<|begin_of_text|>)
📄 EosId: 128001 (<|end_of_text|>) 128008 (<|eom_id|>)
📄 RegularVocabSize: 128000
📄 SpecialVocabSize: 256
💡 Arch: Llama
💡 HiddenAct: Silu
💡 Dim: 4096
💡 HiddenDim: 14336
💡 VocabSize: 128256
💡 nLayers: 32
💡 nHeads: 32
💡 nKvHeads: 8
💡 SeqLen: 131072
💡 NormEpsilon: 0.000010
💡 RopeType: Llama3.1
💡 RopeTheta: 500000
💡 RopeScaling: f=8.0, l=1.0, h=4.0, o=8192
📀 RequiredMemory: 12024248 kB
⭕ Socket[0]: connecting to 10.0.3.184:52415 worker
⭕ Socket[0]: connected
⭕ Socket[1]: connecting to 10.0.1.200:52415 worker
⭕ Socket[1]: connected
⭕ Socket[2]: connecting to 10.0.3.186:52415 worker
⭕ Socket[2]: connected
⭕ Network is initialized
🧠 CPU: avx2
💿 Loading weights...
💿 Weights loaded
⭐ Chat template: llama3
🛑 Stop: <|end_of_text|>
🛑 Stop: <|eom_id|>
💻 System prompt (optional): hi

👱 User

who are you?

🤖 Assistant
I'm LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm trained on a massive d
ataset of text from the internet and can generate human-like responses to a wide range of topics and questions. I can help with things like answering
questions, providing definitions, and even generating text based on a prompt.

I'm constantly learning and improving, so please bear with me if I make any mistakes. I'm here to help and provide assistance, so feel free to ask me
anything!<|eot_id|><|start_header_id|>assistant<|end_header_id|>

It's great to meet you! I'm excited to be here and assist^C