I want the output to stop after a question mark appears in the text. However I want to keep the question mark. I could just append a “?” but that assumes the generation stopped due to a “?” and not some other reason. Perhaps another work around would be to make the “finish_reason” field of the response give the stop sequence that cause the stop. This way developers can do what the wish with that information.
1 Like
I don’t foresee this being implemented at anytime in the near future.
There are basically three termination reasons:
- The response concluded naturally.
- The response triggered a stop condition.
- The response hit the token limit.
If it hit the token limit you almost certainly won’t need or want to append your stop character.
If it concluded naturally, it shouldgenerally be terminated with a punctuation character other than your stop condition and be less than or equal to the token limit, so you won’t need or want to append your stop character.
So, there are ways to determine if the stop condition was met even without OpenAI reporting it in the response.
You could also try making the stop condition ? so the ? shouldn’t get stripped out when it concludes naturally, which may simplify things.
1 Like
I agree that assuming punctuation at the end would help my usecase (as you quite rightly guessed my desired output is natural sentances). However I think its a stretch to say that this is sufficient. Relying on consistency from a stochastic model is never a sound solution particularly for high sampling temperatures.
Lets say I want to extend my usecase a little and stop after a question mark OR an exclemation point OR a period. I now definetly have no work around.
Sure, there’s other things you could try though.
You could do few-shot learning to instruct it to terminate each sentence with an underscore._
Then just make your stop sequence an underscore and you’re done!_
Or, if the “_” doesn’t work for you because it might appear in the response text (especially because it can be used in markdown to typeset italics), just choose another character or set of characters as your termination sequence, maybe even an emoji?
Edit:
1 Like