Trying to call the GPT5 model with the responses API. My system prompt is consistent and is longer than 1024 tokens but I don’t hit the cache for any of the GPT5 model series input_tokens_details=InputTokensDetails(cached_tokens=0) When I use the same for o3-mini and other older models i’m hitting…

Round 1: Standard Chat Models For 3 trials of gpt-4o-2024-08-06 @ 2025-09-23 10:04PM: Stat Average Cold Minimum Maximum stream rate Avg: 57.867 Cold: 69.2 Min: 50.3 Max: 69.2 latency (s) Avg: 0.667 Cold: 0.6606 Min: 0.5961 Max: 0.7443 total response (s) Avg: 2.903 Cold: 2.4962 Min: 2.496…

Same here, it seem that i randomly get cache hits and most cache miss without changing anything. Changing to any other model than gpt5 the problem does not happen . Is there a case because gpt5 is like a model router . It uses different models behind the scenes and that causes a cache miss since it…

After many hours or figuring out what is going on I think I found the solution. I was using an old api version and for some reason it randomly returned or didnt return the cached tokens Switching to OPENAI_AZURE_API_VERSION=2025-04-01-preview did the trick

I’m calling the openai api directly from openai services, are you using models hosted in azure? I tried the latest model string as well gpt-5-mini-2025-08-07 but to no avail

You actually correct after further testing with both azure openai and open ai api keys , the caching seems to be random . What worked for me in azure although pretty weird , I lowered the usage limit and managed to get almost 99% cache hits after the first message. This maybe has to do with the fac…

Update: Using repsonses api rather than completions seems to pretty much solve the problem for gpt5 models

interesting I am using responses API with openai api but still no caching. Are you using the openai sdk for these?

I’m using gpt-5-mini with the Response API from the javascript SDK. I noticed the same issue: Long static system prompt (role=developer), 9,000+ tokens Short varying user prompt (100-200 tokens) The cache hit rate was 1 in 20. And I couldn’t reproduce that one hit afterward. That one hit proved t…

@OpenAI_Support can we file a bug, this is a consistent issue for many users on any model from GPT5 series?

I’m also encountering this issue. No caching at all using gpt 5 nano with responses api.

Caching is borked for GPT 5 models

Prompting

satwik_dai October 22, 2025, 10:36am 13

Facing the same issue for various GPT 5 models.
@OpenAI_Support Could you please address this

Topic		Replies	Views
We need to talk about prompt caching Feedback prompt-caching , responses-api , chat-completions-api	1	783	October 25, 2025
Cache not caching more than 1024 tokens (expected: increments of 128 tokens) Bugs prompt-caching	6	440	November 14, 2024
'OpenAI' object has no attribute 'Completion' API	13	571	January 14, 2025
Assistant API Response is Extremely Slow API assistants-api	1	591	October 21, 2024
OpenAI Why Are The API Calls So Slow? When will it be fixed? API	103	57612	February 19, 2024

Caching is borked for GPT 5 models

Related topics