Have a way to stop text AFTER a stop sequence not before

I want the output to stop after a question mark appears in the text. However I want to keep the question mark. I could just append a “?” but that assumes the generation stopped due to a “?” and not some other reason. Perhaps another work around would be to make the “finish_reason” field of the response give the stop sequence that cause the stop. This way developers can do what the wish with that information.

1 Like

I agree that assuming punctuation at the end would help my usecase (as you quite rightly guessed my desired output is natural sentances). However I think its a stretch to say that this is sufficient. Relying on consistency from a stochastic model is never a sound solution particularly for high sampling temperatures.

Lets say I want to extend my usecase a little and stop after a question mark OR an exclemation point OR a period. I now definetly have no work around.