Documentation is lying or how to understand this?
For gpt-3.5-turbo-1106 it says 16,385 max tokens — https://platform.openai.com/docs/models/gpt-3-5
In reality, we get an error and response that gpt-3.5-turbo-1106 supports only 4096 tokens
16k is the context size limit, including both input + output tokens.
4k is the limit of the output tokens. Make sure you set max_tokens <= 4096 and input_tokens + max_tokens <= 16385 when you call the API.
1 Like
Ah yes, I see, you are right. I found the error in our request