I want the output to stop after a question mark appears in the text. However I want to keep the question mark. I could just append a “?” but that assumes the generation stopped due to a “?” and not some other reason. Perhaps another work around would be to make the “finish_reason” field of the response give the stop sequence that cause the stop. This way developers can do what the wish with that information.
1 Like
I agree that assuming punctuation at the end would help my usecase (as you quite rightly guessed my desired output is natural sentances). However I think its a stretch to say that this is sufficient. Relying on consistency from a stochastic model is never a sound solution particularly for high sampling temperatures.
Lets say I want to extend my usecase a little and stop after a question mark OR an exclemation point OR a period. I now definetly have no work around.
Is there any update to this? Or has the API been updated to acomodate this?
You can add an instruction to the prompt to add some marker after each question mark. Like “?qqq” instead of “?” and use “qqq” as a stop sequence