Many apps pass the same context over and over again to the model, for instance a codebase, product information, a bunch of function calls, or a long multi-turn conversation history. Starting today, we will give you a 50% discount for input tokens that the model has seen recently, no action required.…

Prompt caching (automatic!)

MARK0 October 2, 2024, 9:19am 17

I see. Well, Anthropic allows developers to manage their private prompt caches. I think that it is still an interesting feature so I was hoping to see something similar in OpenAI’s API.

Topic		Replies	Views
Prompt Caching for o3-mini? API o3-mini	3	407	February 4, 2025
How Prompt caching works? API assistants-api , prompt-caching	17	5124	February 4, 2025
Prompt Token Cache Gaming to Save Money? API prompt-caching	1	525	October 18, 2024
Realtime API pricing is wrong, will overcharge API realtime	36	3282	January 15, 2025
Prompt caching with multiple agents API	1	461	October 9, 2024

Prompt caching (automatic!)

Related topics