Large Discrepancy in Output Token Size Between Two Identical GPT-4o-mini Batch Runs

API Bugs
1

Hi OpenAI team,

We are currently using the batch API with the GPT-4o-mini model to identify features in product images.

However, we have encountered a significant issue — for the same input image, prompt, and model, processed via batch API on two different dates (March 11, 2025 and April 24, 2025), we observed a large difference in output token size.

Details:
Model Used: gpt-4o-mini
No change in: Input prompt, image, model, or temperature/settings.
Discrepancy: The output token size between the two dates varies considerably, despite identical inputs.

2

Attaching Batch Output for both dates

3

OpenAI “fixed” their billing, and you got billed the expected amount: vision with gpt-4o-mini is TWICE the cost of gpt-4o.

Your URL: https://d2h50zujfkj84t.cloudfront.net/product_images/Screenshot_2024-02-07_at_7.17.53PM.png

Paste URL into my pricing page to download and calculate cost:

See the calculated expense; see the agreement with your huge and tiny screenshot.

The image shows a code snippet displaying usage statistics with 26,504 prompt tokens, 504 completion tokens, and a total of 27,008 tokens, along with details about cached and audio tokens both set to 0. (Captioned by AI)
The image shows a code snippet displaying usage statistics with 26,504 prompt tokens, 504 completion tokens, and a total of 27,008 tokens, along with details about cached and audio tokens both set to 0. (Captioned by AI)450×272 33.7 KB

After doing so, select other models in the price calculator page’s drop-down and find an affordable performative option. Add multiple images as you’d have in a user message, and have them totaled.

5

Could you direct us to any post or article confirming that OpenAI has resolved their billing issues?

6

You have seen the evidence of “resolving the billing issues” in your latest calls. You are billed the expected price now. There was just a bug or oversight for a short while coinciding with the release of Responses, that was fixed after a few weeks.

Model Base tokens Tile tokens
4o, 4.1, 4.5 85 170
4o-mini 2833 5667