OpenAI team, thanks for the work you’re doing. I’m thrilled with today’s news. I have a few questions about the Assistants API pricing structure that don’t seem to be documented anywhere: When are we charged? Is it upon initiating a run or when adding a message to the thread? How are tokens calcu…

Major +1 here. We absolutely need to understand the costs of calling the API, both in token counts as well as CI sessions, since those are $0.03 each but the API does not tell you when one is created. Also, since the files being uploaded may be dynamic in size (i.e. related to user inputs), and th…

I think files retrieval are billed differently. it is not token-based, but $0.2/gb/assistant/day. The files size is storage size, not retrieval size. That means, for a given day, a given assistant (attached with the same files, files changes count as different assistant, I think), it will only charg…

But, I’m speculating based on how RAG works, they use these docs to pull out text that they past to the model to answer the question which means whatever text they pull out will be billed at tokens. This can be dynamic even with the same files.

That isn’t at all clear to me. I understand they charge for storage, but then when the content from that doc is brought into context then it probably also counts as tokens. There’s just no way to know based on the shape of the API currently.

I read somewhere, perhaps in this thread, that you are also charged for tokens used each time a search requires re-reading the entire document. [2023-11-06_19-50-16] [2023-11-06_19-50-57] So, it appears that we aren’t just talking input/output tokens, but also context (history) as well as docume…

How does token calculation work with knowledge retrieval? +1 for that

What we need is to receive the cost of a given api request, with a breakdown if possible. Retrieval $0.20 / GB / assistant / day Does this mean I’ll be charged 0.20 per day even if I upload a few bytes? Or will this be proportional to the amount of data? Code interpreter $0.03 / session Wha…

Also interested in this. The feature seems really nice, but also a bit of black box. Can’t really deploy it like this, since costs could spiral out of control

Accompanying the roll-out was the apparent need to get rid of easy-to-retrieve token usage reports by the number of calls per five minutes in your account. All you can see is daily tokens. Take a model that costs 1/10th of a penny per 1000 tokens. People have had the assistant agent go crazy with l…

I agree they need to give us a better view into: token utilization RAG utilization costs to I/O, retrieval, etc…

Assistants API Pricing and Token Usage

API

kesar November 7, 2023, 9:15pm 19

How does token calculation work with knowledge retrieval?

+1 for that

5 Likes

Topic		Replies	Views
Assistants API pricing details per message API api-billing	68	42305	January 29, 2024
Assistant API - What are Context Tokens in the Billing calculation? API assistants	25	13305	January 12, 2026
I'm burning through tokens here. What can I do to minimize that? I've included the text of my instructions to my Assistant API	11	2442	November 14, 2023
How exactly do you get charged for using the API for assistants? API assistants-api	34	7912	January 12, 2026
Assistants API token usage and pricing breakdown clarification API gpt-4 , api , assistants	10	10851	February 6, 2024

Assistants API Pricing and Token Usage

Related topics