Responses API high token consumption

API
,
1

Hello All,

Did anyone experience high token consumption when using the Responses API with gpt-4.1-mini?

In my case, I have a prompt which is 1716 tokens according OpenAI tokenizer (see screenshot below)

The image shows a user interface with code and statistics displaying the number of tokens (1,716) and characters (6,738), along with code referencing the 'gpt-4.1-mini' model. (Captioned by AI)
The image shows a user interface with code and statistics displaying the number of tokens (1,716) and characters (6,738), along with code referencing the 'gpt-4.1-mini' model. (Captioned by AI)511×336 12.9 KB

But the Response API is charging me with 9708 tokens (with 5578 of them cached)

"usage": {
    "input_tokens": 9708,
    "total_tokens": 9875,
    "output_tokens": 167,
    "input_tokens_details": {
      "cached_tokens": 5578
    },
    "output_tokens_details": {
      "reasoning_tokens": 0
    }
  }

Any idea why there is so a big difference between 1716 vs 9708 tokens?

2

That little snippet in your first image says it all:

file_search_call

You are passing the file_search results to the prompt, which upon processing expands the prompt significantly.

3

Good catch @mat.eo!

I just removed file_search from the tools list, and still get 11000 tokens.

In my initial example, there was no file_search_call in the output. See the list of output items I got. The number is the message length in characters.

message 149
message 156
message 157
message 164
function_call check_delivery_status
4

You’re using function calling, which also expands the prompt.

All features outside of the system message and prompt use additional tokens. It’s not accurate to copy and paste the code that prepare your request into a tokenizer.

5
  1. The tokens counted isn’t the JSON of a request like you pasted into the tokenizer. It is the language seen by AI.
  2. The cached tokens indicates you are reusing a previous ID and previous chat state, where you pay for all the chat turns seen before, including the tool calls.

Reset the chat. See the token count drop.

Do not use the Responses’ server-side state by response ID reuse, unless you like paying for massive chat cost run-up possible on 1M context input models. OpenAI gives something unsuitable for untrusted users who aren’t paying the bill - manage your own chat history length in coordination with the caching discount and expiry period.

6

How can I reset the chat?
By the way, I am not storing responses on the server. I manually manage chat history.
I have the following in my API call payload:

  "previous_response_id": null,
  "store": false,
7

My guess for these extra tokens is that responses API “thinks” about:

  • should it use file_search
  • should it call some function
  • something else???
    and that “thinking” eats input tokens.
8

Reset, meaning just abandon the chat session, don’t reuse any previous response ID, equivalent to “start a new chat” as a user button (or what the user must do if you enforce a maximum number of turns as your expected service).

If you are in control of all the messages you send, by not reusing a previous ID, then you likely are in complete observation and control of everything you are resending as input messages.

All the input you send is billed again, even if continuing a chat session. To reduce your costs, you reduce the length of history you preserve.

If you are making quick successive calls, at actual chat speed, then you may receive a cache discount of 50% or 75% on that part of input from the start that was identical to a previous API call’s input.