Bug report: gpt-5.6 prompt caching never matches partial prefixes (only exact full-prompt matches)
Product: OpenAI API — Prompt Caching
Models affected: gpt-5.6-sol, gpt-5.6-luna (both confirmed)
Models NOT affected (controls): gpt-5.5, gpt-4o-mini (partial-prefix matching works as documented)
Endpoints: both /v1/responses and /v1/chat/completions
Date of tests: July 14, 2026, ~19:00–20:30 (UTC+3:30) — repeated over ~1.5 h
Org ID: org-XXXXXXXX (fill in from platform.openai.com/settings)
Summary
On gpt-5.6 models, cached_tokens is greater than zero only when the entire prompt is byte-identical to a previously seen prompt. Requests that share a long identical prefix (well above the 1,024-token minimum) but end with a different final user message never produce a cache hit — cached_tokens=0 and cache_write_tokens shows the full prompt being re-written on every request.
This contradicts the documented behavior (“The API caches the longest prefix of a prompt that has been previously computed, starting at 1,024 tokens and increasing in 128-token increments” — [Prompt caching guide]( Promp caching | OpenAI API )), and it makes prompt caching unusable for the standard pattern the same guide recommends: static context at the beginning, variable user question at the end.
The behavior reproduces:
- on both
/v1/responsesand/v1/chat/completions; - with and without
prompt_cache_retention: "24h"; - with prefix sizes from ~3.9k to ~45k tokens;
- with a constant
prompt_cache_key(derived deterministically from the prefix content), request rate far below 15 RPM per key (1 request per ~3 minutes); - at gaps of 3 s, 15 s, 3 min, 7 min, and 15 min between write and attempted read.
Identical requests against gpt-5.5 and gpt-4o-mini behave correctly, so the request shape, org configuration, and key usage are not the cause.
Controlled reproduction (gpt-5.6-sol, fresh never-cached prefix, prompt_cache_retention omitted)
Fixed stable prefix ≈ 3,887 tokens sent as the first user message; only the final user message (“tail”) changes. Same prompt_cache_key on all calls.
/v1/responses:
| # | Tail | Gap | cached_tokens | cache_write_tokens |
|---|---|---|---|---|
| 1 | A (fixed) | — | 0 | 3,887 |
| 2 | A (identical full prompt) | +3 s | 3,887 ✓ | 0 |
| 3 | random (partial-prefix case) | +3 s | 0 ✗ | 3,898 |
| 4 | random (partial-prefix case) | +15 s | 0 ✗ | 3,896 |
/v1/chat/completions (same prefix, same key):
| # | Tail | Gap | cached_tokens | cache_write_tokens |
|---|---|---|---|---|
| 1 | A (fixed) | — | 0 | 3,887 |
| 2 | A (identical full prompt) | +3 s | 3,887 ✓ | 0 |
| 3 | random (partial-prefix case) | +3 s | 0 ✗ | 3,896 |
Call 2 proves the cache entry exists and is readable within 3 seconds — but only for an exact full-prompt match. Calls 3–4 share ~3,887 identical prefix tokens (≫ 1,024) and read nothing, then re-write the full prompt.
An identical run with prompt_cache_retention: "24h" produces the same pattern, so the retention parameter is not the trigger.
Raw usage object from a partial-prefix attempt (call 3):
{"input_tokens":3898,"input_tokens_details":{"cache_write_tokens":3898,"cached_tokens":0},"output_tokens":12,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":3910}
Large-prefix reproduction (gpt-5.6-luna, ~44.8k tokens)
Same experiment with a ~44,847-token stable prefix: every call reports cached_tokens: 0 and cache_write_tokens: 44847, including calls 3 s and 15 s after a successful write of the same prefix.
Production observation (gpt-5.6-sol, real workload)
Our BI application sends a stable ~40k-token warehouse catalog as the first user message (byte-identical across requests — verified by logging a SHA-1 of the prefix, which also serves as prompt_cache_key), followed by a short variable question. Three consecutive requests, same prompt_cache_key (976b9720…), 3–7 minutes apart:
| Time | prompt_tokens | cached_tokens | cache_write_tokens |
|---|---|---|---|
| 19:32:12 | 40,135 | 40,132 (byte-identical repeat of an earlier full prompt) | 0 |
| 19:35:02 | 40,400 | 0 | 40,397 |
| 19:41:46 | 40,548 | 0 | 40,545 |
The 19:32 request repeated an earlier question verbatim in a fresh conversation (full prompt identical → hit, even 12+ minutes after the write). The 19:35 and 19:41 requests shared the same ~40k prefix but had different questions → zero cache read, full re-write each time.
Control results (same request shape, same key derivation)
| Model | Full-prompt match | Partial-prefix match |
|---|---|---|
| gpt-5.6-sol | ✓ hits | ✗ never |
| gpt-5.6-luna | ✓ hits | ✗ never |
| gpt-5.5 | ✓ | ✓ (cached=3,840 on both endpoints) |
| gpt-4o-mini | ✓ | ✓ (cached=3,456 = 27×128, classic 128-token alignment) |
Notable detail: gpt-5.6 full-match hits report exact counts (cached = prompt_tokens − 3, e.g. 3,887 or 40,132 — not 128-aligned), while gpt-4o-mini partial hits are 128-aligned as documented. This suggests gpt-5.6 is serving from an exact-sequence lookup rather than prefix-block matching.
Minimal repro script (Node 18+)
import { createHash } from 'node:crypto';
const KEY = process.env.OPENAI_API_KEY;
const model = 'gpt-5.6-sol';
const prefix = 'STABLE CONTEXT — use ONLY this JSON catalog: ' +
Array.from({ length: 250 }, (_, i) =>
`table_${i} column_${i} description sales orders revenue customer region date amount status`
).join(' | '); // ≈ 3.9k tokens
const cacheKey = createHash('sha1').update(prefix).digest('hex');
const body = (tail) => ({
model,
instructions: 'Return JSON only: {"ok": true}',
input: [
{ role: 'user', content: prefix },
{ role: 'user', content: tail },
],
text: { format: { type: 'json_object' } },
store: false,
prompt_cache_key: cacheKey,
reasoning: { effort: 'low' },
max_output_tokens: 600,
});
async function call(tail) {
const r = await fetch {
method: 'POST',
headers: { Authorization: `Bearer ${KEY}`, 'Content-Type': 'application/json' },
body: JSON.stringify(body(tail)),
});
const j = await r.json();
console.log(tail.slice(0, 20), '→', JSON.stringify(j.usage?.input_tokens_details));
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
await call('fixed tail: reply now'); // write
await sleep(3000);
await call('fixed tail: reply now'); // exact repeat → cached_tokens > 0 ✓
await sleep(3000);
await call(`different tail ${Date.now()}`); // shared prefix → cached_tokens = 0 ✗ (BUG)
Expected on the third call: cached_tokens ≈ 3,9xx (the shared prefix).
Actual on gpt-5.6: cached_tokens: 0, cache_write_tokens: ~3,9xx.
Replacing model with gpt-5.5 or gpt-4o-mini yields the expected partial hit.
Impact
Any application following the documented best practice (large static context first, short variable user input last) gets a 0% cache hit rate on gpt-5.6 and pays full input price on every request — in our case ~40k tokens × $5/M per question that should have been billed at $0.5/M, roughly a 10× overcharge on the cacheable portion. We have temporarily downgraded the affected operation to gpt-5.5 (where caching works) solely because of this issue.
Questions
-
Is exact-full-prompt-only matching an intended (undocumented) limitation of gpt-5.6, or a regression?
-
If intended, could the [Prompt caching guide]( Promp caching | OpenAI API ) be updated to state it, and is a fix planned?
Happy to provide request IDs, org ID (org-AgAtXXXXXXXXXXXIpkeB), timestamps, or run further instrumented tests on request.
