Hello. For the past couple days, GPT-4o input tokens are almost never being cached, despite direct sequential calls with large inputs. When I make the exact same calls with o1-mini, the caching works fine. It’d be lovely to have this fixed and save some dollars. Thank you.

[iScreen Shoter - Safari浏览器 - 250108150122] Since today(2025-01-08), it seems the cached input is back. I’m not sure. Let’s keep watching.

[Screenshot 2024-12-30 at 22.50.48] Exactly same here. What exact date the problem has started? For me it was 17th … 19th… and remains the same till today (January 2nd)

They keep insisting that the problem is with our code. I created a program to list the consumption of all threads and all of them contain cached tokens (about 70% should be cached). and yet there are 0 cached tokens on the dashboard. I’ve had a reasonable increase in the cost of APIs since then and …

Is OpenAI ever going to fix this? This is the same input on another platform. [image]

And this is that same input with OpenAI (not as many times, since caching isn’t working). I have no idea why the tiny sliver of caching occurs. [image]

At least you’re still getting the sliver - I don’t even see that anymore - pretty sure it’s a problem with my code! :smile:

I know I was joking before, but today’s charges were triple what I’m used to – and what I planned on based on caching from a month ago. :grimacing: Is this program you wrote avail for others to use, I’d prefer not to reinvent the wheel to go back through and track everything.

I have the same experience and was wondering where the small amount of caching comes from. I then switched from cost view to activity view in the dashboard. Regarding token usage, it looks almost as if the labels have been switched (and consequently the costs).

I’m seeing the same. Wondering what will happen about the overages these past couple of months.

Same for me. but they didn’t recalculated the past. lets wait

4o input not being cached

API

kmsbernard January 9, 2025, 4:37am 34

Support confirmed that the caching issue is fixed. They’re also looking into any potential overcharges, so past billing concerns might get resolved too.

Dashboard usage vs Prompt response usage not matching

Topic		Replies	Views
Dashboard usage vs Prompt response usage not matching API api-usage , cost , prompt-caching	12	987	January 9, 2025
How Prompt caching works? API assistants-api , prompt-caching	17	10741	February 4, 2025
Realtime API pricing is wrong, will overcharge API realtime	36	5279	January 15, 2025
Prompt caching broken for GPT 5.4 and 5.5 Bugs	4	451	June 13, 2026
Is this a problem with cached tokens? API gpt-4 , prompt-caching	3	1463	October 10, 2024

4o input not being cached

Related topics