I honestly don’t understand ALL of the linked thread on prompt caching either
but I provided it to Claude with your questions…
Scenario 1: Fewer cached tokens than total
- Caching starts at 1024 tokens and increases in 128-token blocks
- Maximum cached tokens will be the largest multiple of 128 that fits your total
- Example: With 5672 total tokens, you’ll see 5432 cached (42 blocks of 128 + 1024)
Scenario 2: Large cache drop with small changes
- KV cache requires exact prefix matches
- Even small changes early in the sequence break prefix matching
- System must find next valid cache point after the change, potentially invalidating large sections
Scenario 3: Token sequence priority
- Caching prioritizes prefix of input (system prompts → tools → user messages)
- Put static content first (system prompts, tools) and variable content last (user messages)
- Cache typically lasts 5-10 minutes (up to 1 hour in off-peak times)