I’m trying to translate text using the API. The source text is about 2000 tokens, but the translation is more than 4096 tokens, and only a part of the translation is returned. I tried different options to get a sequel: I asked for it, I sent the first part with a request. But in response to a repeated request, it simply invents a continuation, instead of translating the rest of the text. Is there any example of implementing such a flow?
1 Like
All new models have a hard output limitation of 4k.
For the case of a language translated and tokenized into a much sparser language encoding, you might consider gpt-3.5-turbo-16k-0613 for the task. It is the next step up that has no limit on the output, up to using the entire context length from input and output.
Automatic chunking is also possible, and will give you higher quality in general when you give the AI less text to follow at once (unless you have specific need for the translation to be highly context-aware).
2 Likes
Welcome to the community @babenkoma
What model(s) are you using for the task?
UPDATE:
Finish reason length simply means that the model doesn’t have any more context length remaining to generate the tokens. Even if you were to ask it to continue, it wouldn’t work and even give you an invalid request error.
FIX:
Use a model with context length greater than 8k like gpt-4, gpt-4-32k, gpt-3.5-turbo-16k or the latest gpt-4-turbo-preview
1 Like