How to use cached_tokens field to calculate cost estimation

[_j]

Let’s just search things I’ve said before…

OpenAI Agent SDK – Token usage and reasoning output

You’ve got it! Prompt tokens and Output tokens are your billing at the model’s rate. Because some output was reasoning output doesn’t change the cost of output tokens. The usage detail can modify the model costs in other categories, though. For example, cached input would provide the portion of the prompt tokens that have been discounted, and audio tokens (chat completions) are the portions billed at the audio rates (approximately 10 times higher).

Some code developed for demonstrating a model call’s final price, by its token costs and the returned usage and its cached_tokens field (not considering audio token costs).

How to get the cost for each api call?

The billing is in terms of tokens. You can then directly translate that to the input and output cost of the model you are employing - the cost being divided by one million then multiplied by your usage. Just typing up a request to an API endpoint.. ... import os, httpx >>> def send_chat_request(conversation_messages): ... # Set your API key in OPENAI_API_KEY env variable (never hard-code it). ... api_key = os.environ.get("OPENAI_API_KEY") ... if not api_key: ... raise Value…

Note that “cached” is not ultimately as a percentage, it is a reduced token cost specified per-model, and before discount percentage varied, that code snippet anticipated this by taking the model pricing fields.

OpenAI makes a “best effort to route to the same server”, but the context window cache can persist for a time as low as five minutes. There isn’t a central cache database, it is per AI model server.

How to save input tokens in Responses API?

To receive a (non-guaranteed) discount, the entire initial input context needs to be the same. The cache is temporary after the recent call, with a lifetime around 10-60 minutes (someday I’ll classify how long is delivered). If you change initial system message instructions from “you are a helpful ai” to “you are a helpful ai with these user preferences”, then you have broken the caching. That also applies to the list of internal tools and developer-provided functions that are appended to your…

You can read details about how the same server is “found”…

Prompt Cache Routing + the `user` Parameter

We just updated our prompt caching guide with details of how cache routing works! We route prompts by org, hashing the first ~256 tokens, and spillover to more machines ~15 RPM. If you’ve got many prompts with long shared prefixes, the user parameter can improve request bucketing and boost cache hits [This image explains the automatic caching process for API prompts of 1024 tokens or longer, detailing steps including cache routing, lookup, cache hit, and cache miss, with specifics on how requ…