Hello. For the past couple days, GPT-4o input tokens are almost never being cached, despite direct sequential calls with large inputs. When I make the exact same calls with o1-mini, the caching works fine. It’d be lovely to have this fixed and save some dollars. Thank you.

[iScreen Shoter - Safari浏览器 - 250108150122] Since today(2025-01-08), it seems the cached input is back. I’m not sure. Let’s keep watching.

I haven’t received any updates from support yet. It’s getting really frustrating.

I haven’t gotten any feedback yet. I’m experiencing a considerable increase in the cost of using the API and I’m worried about the viability of the operation if it continues like this.

Has anyone gotten an acceptable response to this problem from support other than “Check your application because you’re probably doing something wrong.”?

:smile: I got this support request so often in the last 3 months I just gave up reaching out to them (Hint: you’re not doing anything wrong). Also, I think they’re off for 2 weeks so you might not even hear anything until next week (if at all about this…)

Similar issue here. We track our tokens from each request internally. While we are no longer seeing cached tokens, we are also seeing an increase in token usage in our dashboard that is not correct. We’ve notified OpenAI support, but I recommend you log your own usage as well. We are being charged nearly double the price.

[image] I just received this response.

[image] Similar issue here, started from ~18 Dec. Also we monitor cache by analyzing the usage field in completion responses. Our current metrics show approximately 80% cache hit ratio, while it’s <1% in OpenAI dashboard. Haven’t got any response from OpenAI support yet

[Screenshot 2024-12-30 at 22.50.48] Exactly same here. What exact date the problem has started? For me it was 17th … 19th… and remains the same till today (January 2nd)

They keep insisting that the problem is with our code. I created a program to list the consumption of all threads and all of them contain cached tokens (about 70% should be cached). and yet there are 0 cached tokens on the dashboard. I’ve had a reasonable increase in the cost of APIs since then and I believe I’m being extra charged for this. [image]

Is OpenAI ever going to fix this? This is the same input on another platform. [image]

4o input not being cached

API

b.silva December 31, 2024, 2:37pm 22

I just received this response.

Dashboard usage vs Prompt response usage not matching

Topic		Replies	Views
Dashboard usage vs Prompt response usage not matching API api-usage , cost , prompt-caching	12	1223	January 9, 2025
How Prompt caching works? API assistants-api , prompt-caching	17	11533	February 4, 2025
Realtime API pricing is wrong, will overcharge API realtime	36	5740	January 15, 2025
Prompt caching broken for GPT 5.4 and 5.5 Bugs	4	581	June 13, 2026
Is this a problem with cached tokens? API gpt-4 , prompt-caching	3	1520	October 10, 2024

4o input not being cached

Related topics