Implement the a way to "continue generating" with the API

I think it would be awesome to have an easy way (with the API) to continue generating when the generation stopped for length reasons.

For instance, I’m translating a very long json file, and it seems very complicated (even impossible?) to continue the generation where it stopped. With the user interface, the “continue generation” button made it possible.

I can see that ChatGPT UI uses a custom API that takes an “action” parameter (which value is “next” or “continue”) along with a conversation identifier. Could it be possible to have the same with the Chat completion API?

Thanks for your great work

13 Likes

I’m also waiting for this problem, many of my requests are timed out because no API feedback on this feature and in fact, I can’t get any results, and the tokens are counted down, I put everything on hold in waiting for a solution…

2 Likes