Are API calls free of charge?

API
1

I have a question about OpenAI API pricing structure. I understand the token-based pricing, but I’m wondering about the relationship between API calls and total cost:

Is there a difference in cost between these two scenarios?

  1. Making a single API call that processes 1,000 tokens
  2. Making multiple API calls that total 1,000 tokens (e.g., 10 calls with 100 tokens each)

I noticed that GPTForWork’s pricing calculator includes a separate ‘Price per API call’ metric for OpenAI, which made me question if there’s an additional cost per API call beyond just token usage.

2

Welcome to the community!

In terms of cost, there’s no real difference for the most part. It’s possible that there might be a caching consideration if you wait more than a few minutes between calls (https://platform.openai.com/docs/guides/prompt-caching), and it’s likely I suppose that requests sent at the same time might miss each other’s caches, so if you want to hypothetically optimize for cost you’d send multiple requests that start with the same context in a sequential fashion.

There’s also the batch API (https://platform.openai.com/docs/guides/batch) where you have to send all your stuff at once, for all intents and purposes, to get a discount.

Another consideration might be the request rate limits. Obviously, a request is a request, and a factor 10 increase might not scale so well. But that depends on your tier and use case. (https://platform.openai.com/docs/guides/rate-limits#how-do-these-rate-limits-work)

The last thing that comes to mind (but you probably know this) is that each request prices the context again. Apart from using caching (as above), if you have a massive context and can perform all your generation tasks in one shot, then that saves you from paying 10x the context cost (or 1+~9*0.5, whatever the case may be with caching.)

About that product you mentioned: It looks like it’s just an api wrapper - what these people do for their own cost tracking doesn’t necessarily have to have anything to do with OpenAI pricing.

But as you can tell, the TL;DR: is, it’s complicated. But not that complicated. But sometimes difficult to predict.

2 Likes
3

So answering to the original raw question, there is no additional cost for making the API request.

Quick note - Now I see that the tool I was mentioning was just a wrapper instead of a calculator.

1 Like
4

There are some new per-call charges to note that are in addition to the token consumption prices of models.

Responses endpoint:

  • web search tool
  • file search tool using vector stores

Chat completions:

  • web search AI model

Assistants:

  • code interpreter session

File search API:

  • independent use of search
  • (data storage is billed hourly for daily rate by amount you persist in vector stores.)
2 Likes