Sometimes the completion gets stuck repeating itself until the MAX_TOKENS is reached. Is there a good way to stop this from happening?
I tried to play with presence_penalty and frequency_penalty but it sometimes still gets stuck in repetition mode…
The best option is to use the stop setting, but it depends on what you want to achieve
For example, if you want to stop after a paragraph, you can use the ‘\n’
If you trained the model, you can include end markers in your completion. Something like “{your completion} END”
Then your stop is the word “END”
Presense_penalty should stop is reusing words/tokens as well (Not this is individual token/word reuse - not entire sentences. It may reword the sentence still)
1 Like
Indeed the stop setting seems to be the solution here.
It seems to work with ‘\n\n’.
2 Likes