I’m new here, and I’m working on integrating with Assistants using Assistants API. The criteria of getting charged still confuses me.
I created an assistant in the playground, with specifying the instructions, are the instructions being sent with the message each time I add a new message to the thread? If yes, does that mean that I’m paying for the instructions for each message?
Let’s say I sent a message A and got a response B, when I send a message C, I will be billed for number of tokens in messages A and C as an input, and number of tokens in messages B and D as well as an output, but is there any extra charges?
It is mentioned that using gpt-4 in the assistants costs 0.20$/GB per day. Can I say that the daily total: 0.20$/GB per day + number of tokens in messages A, B, C, and D?
Sorry I’m really confused in figuring out the pricings.
The instructions you include are just part of other instructions unseen that are indeed sent to the AI model every call. But more importantly, when using any of the functionality besides just the instruction, the AI agent has the ability to make multiple repeating iterative calls that each cost in tokens.
Then when the conversation grows, there is no truncating it, each chat turn, each assistant response, the inputs and outputs of functions and retrieval all load the AI context to maximum.
There is absolutely no accounting for or predicting how high a single user input could potentially be.
You must only use the AI model GPT-4-1106-preview, or gpt-3.5-turbo-1106, as they are the only one that can emit parallel tool calls required by retrieval (no, you don’t get embeddings RAG for free), to do otherwise can have costly failure loops. They will also currently fail on functions using accented characters in some languages and unicode.
The cost you refer to though is the cost of data storage, multiplied by the number of unique assistants linked to that data storage, per day.
And no, there is no report for you when all is done with your usage. All-in-all: avoid.