Llama Ignoring Reverse Prompt Every Other Time

[loukylor]

Hi, I'm sure this is a relatively easy fix, but I have an issue where the AI will ignore the reverse prompt in interact mode every other time:

You can see in the image it writes User: (which is my reverse prompt) but still continues to write without any input from me, where it responds again, then finally stops when the prompt comes up.

Here is my full command line: main -m ./models/13B/ggml-model-q4_0.bin -n -1 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt --in-prefix " " -c 2048 -t 8
The prompt is the chat-with-bob.txt from examples.

[klosax]

I also had this issue, but got it to work better if I didnt use the --in-prefix parameter and skipped the whitespace following the names. It seems that the reverse-prompting wont work well ending with whitespaces.

User:Hello, Bob.
Bob:Hello. How may I help you today?
User:Please tell me the largest city in Europe.
Bob:Sure. The largest city in Europe is Moscow, the capital of Russia.
User:

[loukylor]

According to the docs, I'm using the --in-prefix param properly, so perhaps this is an actual bug and not just incorrect usage?

[ivanstepanovftw]

Please, share full output including seed, so we can reproduce it

[DannyDaemonic]

I just completely redownloaded the 13B and 7B models

Sorry. You didn't have to do that! To rule out corruption, there's a checksum included in the source, you could have compared it to the checksums there.

So I think it is just something wrong with reverse prompt. Should I make an issue on the repo?

Yeah, be sure to include a sample output made from a model that matches the checksums. If your quantized versions don't match use an f16.bin. (I don't know why none of my quantized models match any of the ones in the sha256sums file. I rebuilt them recently.)

Might be a good idea to include what OS you're on, because I'm having trouble repeating this on Linux. I don't know how to show what version you're using but maybe just update to the latest version to ensure it's still happening.

[loukylor]

I include the info in the issue I just made, but I'm on Windows 10 and I'm on the latest commit (as of about 30 minutes ago).

[DannyDaemonic]

I just saw it. Thanks for posting. It looks good. Probably not too important, but they use sha256sum for the checksums, not md5sum.

[loukylor]

Oh, I just followed what was in the issue template. Might be worth it to pr to change it to sha256sum. Also thanks, no problem.

[DannyDaemonic]

Oh yeah, thanks for reminding me about that. I prefer md5sum myself but I'm guessing they've switched over to using sha256sum since that's the happyface standard and a lot of people are just downloading the models from there. I'll put a PR in to change it and see what they say.