**Subject:** Issue with Token Limit for `gpt-4o-mini` Model in `v1/chat/completions` API

Documentation
1

Description:*

I have been testing the gpt-4o-mini model using the v1/chat/completions API and have encountered an issue regarding the token limit. According to the official documentation, the context window for the gpt-4o-mini model is specified as 128,000 tokens. However, in my tests, the total token length limit seems to be restricted to approximately 16,000 tokens instead.

Here are the details of my test:

  • Model: gpt-4o-mini
  • Test Scenario: I included a history of around 16,000 tokens and set max_tokens to 4,000.
  • Issue: The response was truncated, and I only received a few hundred tokens of output. The stop_reason in the response indicates that the length limit was exceeded, despite the remaining difference between 16,000 tokens and the documented 128,000 tokens being substantial.

I would appreciate it if you could investigate this discrepancy and confirm whether there are any additional limitations or if this might be an issue with the API. Your assistance in clarifying this matter would be greatly valued.

Thank you.

2

Welcome to the dev forum @monkeydk6666

The context length of 128,000 means that the model can process a maximum of 128,000 tokens (input + output).

16K is the maximum output token limit, i.e., the model output is capped at 16,384 tokens.

1 Like
3

But in my actual usage, the input plus output is only 16K.

4

In that case you don’t need to set a value for max tokens. The model will use the max output length available to it and if your output is less than 16,384 token output limit, you’ll get the complete output in one API call.