Generation will also stop with the API if the request produces more tokens than there is room for in the reply. you can always prompt the model again to continue from where it left off if this happens and you will get a finish_reason of “length” so you can detect when this happens.
2 Likes