The API throws an error when the context length is at the maximum.
The limit was 4096 before, now it’s 4097.
It doesn’t matter what the system_prompt, max_tokens or user prompt are.
As soon as the total num_tokens_from_messages equals the maximum, the API throws an error.
Tested with model = “gpt-3.5-turbo”.
The workaround is easy, just make sure you don’t exceed 4096 tokens. But it appears that there’s an indexing error or comparison error in the API.
I had the error before with 4096 tokens (and reported the bug last week). It was driving me crazy, but now I started testing again, and the ‘workaround’ of OpenAI to raise the limit by 1 will help with my old code, but it’s still off.
Another question: is the cookbook’s example for num_tokens_from_messages at How to format inputs to ChatGPT models | OpenAI Cookbook still up-to-date with the latest version of gpt-3.5-turbo, or has it changed? Because from my testing it calculates exactly 10 tokens less than expected, for a complete messages object.
Error message:
openai.error.InvalidRequestError: This model’s maximum context length is 4097 tokens. However, you requested 4097 tokens (1647 in the messages, 2450 in the completion). Please reduce the length of the messages or completion.
Example message below, if you remove one number from user content or two words from system content, it will work again:
{
“model”: “gpt-3.5-turbo”,
“temperature”: 0.2,
“max_tokens”: 16,
“messages”: [
{
“role”: “system”,
“content”: “I am system prompt”
},
{
“role”: “user”,
“content”: “4097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974097409740974”
}
]
}