How to generate a fix amount of tokens when calling the model #1334
Unanswered
JeremyGe07
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have read the doc but sitll cannot find how to generate a fix amount of tokens in call function. Lets say, e.g., how to gurantee to generate 50 tokens in the following code?
In llama.cpp ,we can use the -n(--n-predict) parameter to set the number of tokens to predict when generating text, and use the --ignore-eos parameter to keep generating until the set nubmer length as this doc said.
Thanks for your help in advance.
Beta Was this translation helpful? Give feedback.
All reactions