hey there, we would like to understand the details of assistants API pricing. For the assistants API, there are 3 parts: instructions previous messages inside the thread new message for the thread Whenever I send a new message, will my token consumption include 1. instructions? 2. previous messa…

the assistants API just uses the existing models - all three are considered messages, with each of them having role (instruction should be system, also there’s user, function and assistant) and content (functions further have name). the tokens of each message’s content value is considered. And becau…

[image] sam: And because you’ll probably send the whole conversation, including instructions etc., to the API every time you expect a reply (message with role “assistant”) - you’re being billed for all of them over and over again; this is, unless they are trimmed. Ya as far as I can tell this…

[image] sam: you should programmatically trim them or cap the context size (depending on your needs) for pricing to not explode - if a long gpt-4 thread has the theoretical max content size of 128k tokens, each new interaction with the thread could become expensive. Thank you for this note. …

@sam I’m confused about the assistant API costs. If we incrementally add messages to the thread, we pay for those tokens. Why do we need to resend the whole conversation to the server over and over for each user interaction? I was hoping using threads could reduce our costs of keeping the history…

I don’t think that’s how it works. From what I’ve seen, the server doesn’t have persistence (otherwise they would have said so at the conference). That’s just how the API is laid out. The Assistants suite is just a way for not so tech-savvy users /devs to implement GPT into their application.

You’re right that you’re only sending the new messages to the thread but all the other messages are being stored serverside and are used when you run the assistant. All the previous messages in the thread + your new one are then run through GPT4+ so you’re paying for all the tokens when you run it, …

Can you cap the context size inside the assistant dashboard ?

Yeah, this is really annoying. To me it is nonsense to pay for the context that I have already sent over and over.

It’s not about sending, it’s about how it has to use those tokens in the model again which is what you’re paying for. The more tokens in the model, the harder and longer it takes so it cost more.

You could delete older messages from the thread to limit the context size if you wanted to

Assistants API pricing details per message

API

bloodlinealphaDev November 7, 2023, 3:45am 3

Ya as far as I can tell this is what is happening . Hopefully there is way to control this a bit more, as the cost for my simple assistant was roughly $0.90 (GPT4-1106) while only asking less than 10 questions. This is not really economical to use all the time…

Topic		Replies	Views
Assistants API Pricing and Token Usage API api , pricing	103	34770	February 9, 2024
How exactly do you get charged for using the API for assistants? API assistants-api	33	8071	November 27, 2023
I'm burning through tokens here. What can I do to minimize that? I've included the text of my instructions to my Assistant API	11	2552	November 14, 2023
How to accurately price a gpt-4 chatbot? API gpt-4 , api	63	26236	December 6, 2023
Assistant API - What are Context Tokens in the Billing calculation? API assistants	24	13507	May 6, 2024

Assistants API pricing details per message

Related topics