Hi everyone,
I’d appreciate your help. Could someone please clarify the maximum number of tokens in the context window for the babbage-002 model? The Models page states that the maximum tokens are 16,384, while the Fine-tuning page says, “The legacy prompt completion pair data format has been retained for the updated babbage-002 and davinci-002 models to ensure a smooth transition. The new models will support fine-tuning with a 4k token context …”, which seems to imply a 4k maximum context for the babbage-002 model. Meanwhile, the Playground limits the ‘maximum length’ to 2049 tokens for babbage-002.
So, which one is it?
Welcome to the Forum!
Given that it has been specifically addressed in the fine-tuning Q&A, I would go with the guidance from there.
I believe there is an issue/glitch in the playground as the maximum token value seems to have a different meaning in the playground depending on the selected model - so would not consider this a reliable source at the moment. What I assume is that in the case of babbage-002 the 2,049 token limit in the playground refers to the maximum output tokens.
Playground states that a maximum of 2049 tokens are allocated for both the prompt and response together for the babbage-002 model. Could the context window be 4k for fine-tuning and 16k for the API? A clarification from OpenAI regarding the different maximum context lengths used for the babbage-002 model in various parts of their documentation and Playground would be appreciated.
Yes, that’s my interpretation of the information as well.
Agree it could be made clearer across the different docs and platforms.