Heyy, can someone list the INPUT token limit for each of the gpt models in api
GPT-4 TURBO
GPT-4
GPT-4o-mini
GPT-4o
GPT - 3.5 Turbo
Can you please list the input token limit for all?
Here’s a listing of all OpenAI models.
https://platform.openai.com/docs/models
A language model has a context window. This is the working memory for tokens where both the input is placed, and after that, more completion tokens are generated.
Being a shared area for both input and output, the limit of input is an amount sent where you still have enough space to form a response in the context window. Therefore, on a model with a 16k context window length, you would not want to send more than 14k if you still want to leave 2k as an area to form a response.
You can use the API parameter max_completion_tokens to artificially limit the amount the AI will produce before a shutoff. This can also act as a “reservation”, because if you request a 4k response before shutoff (a number similar to the artificial limit OpenAI placed on many models), the rate limiter will deny the API request if the input does not allow this much output to be generated in the remaining context window length.
Other per-minute rates at low payment tiers may simply block API requests with large input if the one request has more input tokens than allowed rate per minute.
1 Like