You want to know what has been completely wiped from existence in documentation?
Any mention of 128 token increments of matching.
There now:
In-memory prompt cache retention (ed: not the 24 hour version that has now been switched on retroactively as a default) is available for all models that support Prompt Caching, except for
gpt-5.5,gpt-5.5-pro, and all future models.
So, have they cheapened out in preload by not producing incremental k-v states?
Is there only one “hash match” for an input, the latest, or the longest? It says:
Extended Prompt Caching works by offloading the key/value tensors to GPU-local storage
You send 272k of input - Is OpenAI now avoiding storing 2100 cache window state increments, replacing that with just one possible match?
The documentation has been altered. The methodology has been obfuscated. The models have been changed. The expected pricing is not being delivered in branches, in parallel calls, in partials that should match.
Feb 10
Today
OpenAI knows what they did. The silence, nothing to diagnose, is evidence.
The only thing that would seem to work would be continuously growing on a prior exact input. It’s not really breaking a promise when the prior operation method and its documentation has been eradicated.
For a prior model than gpt-5.5, you have the option of sending “in_memory”, and must.
This pattern seems to allow and anticipate and only be built for ChatGPT’s use, disregarding API developer’s potential reuses.
There is no good method of making a “mid-state” with a prompt cache key API parameter. That can only serve to break a cache - you can’t change it without a 0 hit being made.
The pattern fits: single growing converations unique to a user. Which cannot be rejoined from an earlier state.
Perhaps the reasoning there are multple feature gates being A/B on ChatGPT users, to turn off message editing and branching in ChatGPT in different ways.