Hello Everyone,
I believe there is a significant issue with the GPT-4o-mini API when using the Vision feature in Batch Processing.
A few days ago, I processed a batch of images with the detail: high setting. At that time, the prompt token count ranged between 300 to 500 tokens per image, which was expected.
However, on March 20, 2025, and in the days leading up to it, using the exact same images (512px width, detail: high), the prompt token count unexpectedly spiked to 8,000 to 14,000 tokens—far beyond the expected range.
I carefully compared the input and output files from March 15, 2025, with those from March 20, 2025. I found no changes in the image format, metadata, or verbosity of the responses.
Summary of Observations:
- Identical Inputs: The same image files were used, with no modifications.
- Drastically Different Outputs: Prompt token usage increased nearly 20–30x.
- Impact: This behavior significantly affects both performance and cost.
Could you please investigate this issue and clarify why the token usage has surged so drastically for identical inputs?
Below are sample records from both dates for your reference:
03/15/2025
{"id": "batch_req_67d50478d8a48190938204745415d924", "custom_id": "row1col1", "response": {"status_code": 200, "request_id": "8149658b170042b57bba8b123386f32e", "body": {"id": "chatcmpl-BBDZqmirdRdsZ7SMrXnfqP2xwCfOZ", "object": "chat.completion", "created": 1742013498, "model": "gpt-4o-mini-2024-07-18", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Bright light floods a spacious living room through large windows that offer a breathtaking view of the city and water. A sleek, modern sofa invites you to relax, paired with colorful cushions that add a playful touch. Nearby, a stylish coffee table holds fresh flowers and interesting books, creating a welcoming vibe. Textured walls bring depth to the space while unique art pieces spark curiosity and conversation. The gentle glow from elegant lamps casts warmth across the room in the evening. Overall, this area feels like a perfect blend of comfort and contemporary design, making it an inviting spot for gatherings or quiet moments alike.", "refusal": null, "annotations": []}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 388, "completion_tokens": 119, "total_tokens": 507, "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0}}, "service_tier": "default", "system_fingerprint": "fp_7fcd609668"}}, "error": null}
{"id": "batch_req_67d50478eab08190955a5bd635713fd4", "custom_id": "row2col1", "response": {"status_code": 200, "request_id": "e054699ad11736d77ad7df0778c49196", "body": {"id": "chatcmpl-BBDaFt7DTZzLLMiaOTsEvLWRGB1go", "object": "chat.completion", "created": 1742013523, "model": "gpt-4o-mini-2024-07-18", "choices": [{"index": 0, "message": {"role": "assistant", "content": "A sleek black wall sets a bold backdrop for this stylish living room, bringing a modern edge to the space. A bright red artwork pops above a chic black couch, adding an inviting touch. Across from it, a comfortable checkered ottoman offers extra seating while tying in playful patterns. A small Christmas tree adds a splash of green and festive charm, creating warmth during the holidays. Soft lighting gently highlights the textures of the furniture and accessories, making everything feel welcoming. An elegant glass chair complements the dark wood elements, pulling together a harmonious blend of materials and styles throughout the room.", "refusal": null, "annotations": []}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 388, "completion_tokens": 117, "total_tokens": 505, "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0}}, "service_tier": "default", "system_fingerprint": "fp_7fcd609668"}}, "error": null}
03/20/2025
{"id": "batch_req_67dbd2ca31348190865937be0733e632", "custom_id": "row8col1", "response": {"status_code": 200, "request_id": "0d620686f36e704fd3ed1e9dc42ed2a7", "body": {"id": "chatcmpl-BD5cfgZoLffagVekNYaRopXcRldsb", "object": "chat.completion", "created": 1742459577, "model": "gpt-4o-mini-2024-07-18", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Bright, airy spaces welcome everyone in this lovely living room. Tall windows fill the area with natural light, showcasing elegant round designs that add charm. A grand piano sits invitingly in one corner, ready for a delightful tune. The coffee table features unique geometric shapes and is adorned with fresh greenery and candles, creating an inviting centerpiece. Soft furnishings in neutral tones provide comfort while art pieces on the walls bring character to the space. Elegant lamps stand tall beside plush seating, enhancing the warm atmosphere that feels both stylish and homey.", "refusal": null, "annotations": []}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 14300, "completion_tokens": 106, "total_tokens": 14406, "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0}}, "service_tier": "default", "system_fingerprint": "fp_3267753c5d"}}, "error": null}
{"id": "batch_req_67dbd2ca43188190bbf0329e43419fd3", "custom_id": "row9col1", "response": {"status_code": 200, "request_id": "8ad2fad37486dfb3e24308aea84812b8", "body": {"id": "chatcmpl-BD5cf8Zqa9av4PQdaVD46g1lBAoA9", "object": "chat.completion", "created": 1742459577, "model": "gpt-4o-mini-2024-07-18", "choices": [{"index": 0, "message": {"role": "assistant", "content": "A lovely gray wall sets a calm backdrop for this inviting space, where a stylish green sofa sits comfortably. A round coffee table made of light wood holds a cheerful potted plant, adding a touch of nature to the room. Nearby, a dining area features simple wooden chairs and warm pendant lights that create an inviting glow. Textures play nicely together, with plush cushions on the couch and a textured rug underfoot. Decorative accents like modern art pieces and unique accessories bring character to every corner. Large windows let in natural light, brightening up the whole atmosphere and making it feel alive.", "refusal": null, "annotations": []}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 8633, "completion_tokens": 118, "total_tokens": 8751, "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0}}, "service_tier": "default", "system_fingerprint": "fp_3267753c5d"}}, "error": null}
{"id": "batch_req_67dbd2ca574081909a1cb08073f90409", "custom_id": "row10col1", "response": {"status_code": 200, "request_id": "47e02b90af66d9aeadd935f0dd8035a8", "body": {"id": "chatcmpl-BD5cfbGLov9Rjg3dkb1o7ybCSgPUU", "object": "chat.completion", "created": 1742459577, "model": "gpt-4o-mini-2024-07-18", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Light gray walls create a calm backdrop for this lovely living room. Two stylish armchairs sit invitingly around a unique coffee table that draws the eye with its interesting shape and material. A large mirror above the fireplace adds depth and sparkle, reflecting warm light from elegant lamps on either side. Decorative pieces, like fresh flowers and artistic accents, bring life to the space. Textured rugs underfoot add warmth, while colorful artwork on the wall introduces playful touches. This room feels welcoming and ready for cozy gatherings or quiet moments of relaxation.", "refusal": null, "annotations": []}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 8633, "completion_tokens": 107, "total_tokens": 8740, "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0}}, "service_tier": "default", "system_fingerprint": "fp_3267753c5d"}}, "error": null}
Thank you in advance for your attention and support. I appreciate any insights you can provide into this issue.
Best regards,
Sam
Solved