Currently, I see the simplest course of action as registering an OpenAI account for each client, which they can top up themselves. My application would then be offered on a subscription basis. However, from a service perspective, this is not convenient.
Otherwise, as far as I understand, I would need to calculate how many tokens they use, how much it costs, etc. But I don’t see the point in doing this. Since OpenAI already does this, what I calculate may not correspond to what OpenAI calculates.
In fact, OpenAI already does all these calculations, and what I need, they can very easily implement. Just have to wait?
well, the way it works is that when you have a conversation, you append the new messages to the old message list and send the whole shebang against the AI again. so you do have a square growth in input token cost, the longer the conversation gets.
if you use streaming, it won’t compute that for you, but you can use tiktoken to get a really good estimate.
edit: I’m wondering, did you choose assistants because you didn’t know you could do it with chat completions? The cost would be similar or less by using the chat completion api directly.
Yeah, but it’s extra complicated with threads. I’m not sure if it still does auto truncation and it depends on if you use runs, but in general there’s a lot of debate around cost control with the assistants.
OpenAI will probably eventually release tools that help you track this (unless they decide to retire assistants completely)
In my biased opinion, the decision to use assistants in a product comes with a ton of risks and not many rewards. You can achieve the same thing with other tools, but I understand that it may be easier for novice developers to get started by just using assistants.
Thank you for opening my eyes to the real pricing of the API. Before this, I thought I was only paying for the message and response. I read this topic: “Assistants API pricing details per message” and i’m outraged.
Probably I will postpone my idea until better times.