I was looking at my bill today and there was a cached input. What is it for? What does it store? How do I cancel it? How do I update it?
Hi @anne.xing,
Prompt caching automatically kicks in for identical (partial or complete) inputs over 1024 tokens sent to the supported models.
It’s actually saving you money as cached inputs cut costs by 50% for long prompts.
More info in the prompt caching guide.
2 Likes