Issues with gpt-5.6 model cache writes and cache retrieval discounts
- broken expectation that any written cache would be retrieved always
- broken expectation that any “implicit” call would be stored for cache if not a hit
- (any expectation of arbitrary cache matches was already broken with 5.5)
What works as a necessary pattern
First, a demonstration of the new per-message-part parameter in play: Here is what is working, after a new UI presentation of prompt_cache_breakpoint: { "mode": "explicit" } as “cache write” (although a more accurate option might be named “cache breakpoint”)
prompt_cache_options - mode: explicit is used (rephrased: turn off the single automatic full-length cache write.
A write at just a developer message sent - and marked “explicit breakpoint”
Then a growing chat on just the stored developer cache being reused.
(developer message is minimized in the screenshot’s GUI, but has text length producing the 1125 token input billing)
Success is also achieved restarting from just the developer message that had message content part marked with an explicit write. Note that the “breakpoint:explicit” per message persists sending “explicit”, but does not force a write with a cache hit.
Unexpected Result
Turn off the particular developer message explicit write - you still have a cache, right? Wrong.
- no cache write (correct)
- no cache discount (wrong)
cached: 0:
How about a complete switch to implicit mode, then?
Wrong: there is a stored cache, the same prompt_cache_key, the same developer message, yet STILL no discount, no hit, and WORSE, a charge for what should have been a hit, and worse a charge at a 0-shot input that will not be a reuse point.
Confirming the cache is still there, check the UI box for developer message and run an original input in “implicit” mode (no top level prompt_cache_options - mode: explicit)
Issue 1
The API is only matching an explicit point already marked and then still marked.
There is no hit on identical input if the message part is not again explicitly marked.
Unexpected: the only way to get a “read” is to mark the same “write”.
This means you are somewhat powerless over replays and refreshing and recalling at a system message, because:
- only the final four explicit marks in an input will be used as potential writes in a new API call (and thus potential read sites also).
- thus, you can mark all the followup user messages of perhaps 6 in-context training examples - including the first “developer” that would have a hit because you constantly reuse it. But the “explicit” is dropped on the most-reused developer message across an app and you get no hit, marked or not - and a full bill for a full write that may be useless.
Then, there is this HUGE issue. Running in “implicit” mode, running only a developer message, expecting it to be stored: there is NO cache write and only full billing:
(note: a complete “reset” by new key to ensure a discrete evaluation)
That means: you expected to warm up the API by simply making a “developer” call. However, no cache was written on a developer message, when it was clearly capable if set “explicit”.
Effect: you thought you stored a point to grow 0-shot turns from. Instead, you got no write and all of your 0-shots are billed at 125% instead of 90% discount of the developer message.
Issue 2
Developer message is not getting a write, even if it is the last. OpenAI is discriminating by role type. A developer message may also be a pre-prompt or post-prompt that is retained in growing chat - yet they ignore the cache write at the full length of input as documented