How not to cut the proposed sentences

during completion some time sentence is not completed like
my name is anish jain and I a
because of the max_token limit but in production we have to use max token limit is there is any way so user get the completed sentence every time they call the api

Hey that was just an example what about in general when we a complete sentences in 1 api call and yes we can use β€œ.” As stop seq but that restricts us for generating only 1 sentence at a time

But how we know its not the completed sentence

You can also use something in a a programming language like JS: String.prototype.includes() - JavaScript | MDN to see if your sentence includes a β€œ.” or a β€œ?”

1 Like

If it’s important, I turn up the max tokens count and then use better stops.

You could also use CURIE-INSTRUCT to ask β€œIs this a complete sentence?” to double check.

This leads to the possibility of a validation endpoint. But you can also check with basic things like regex.

1 Like