When you create an app using openAI APIs, you may want to charge for that.
You have to charge at least the openAI charge.
Is there any way to know the cost on the API response.
Of course, I can know that from my user board. I want to have an automatic way to charge the user for using my app, as I will not have to pay myself.
I am using an Assistant, and I have seen how tricky is the cost. I was thinking an automatic response from them where you see the cost.
Hi!
You want to know the exact cost that have been caused by a specific API call.
We cannot exactly know the cost in advance because the number of output tokens is not known or maybe your app is based on the Assistants API and can perform additional actions under the hood.
My idea is to leverage the changes in rate limits before the API has been called and after the return has been retrieved. You can get the rate limits in the headers and use this data to calculate the costs according to the pricing page.
Initially you would need to know your current rate limits x-ratelimit-remaining-tokens and then get the delta after the reply has been received.
Since the rate limits reset you would need to account for that as well.
https://platform.openai.com/docs/guides/rate-limits/rate-limits-in-headers
It does have to be in advance, at least after it is completed, as I have on my board.
Unless you set-up the system to always reply with a fixed number of output tokens then nobody knows the exact cost in advance. This is due to the stochastic nature of the model. But it’s a possibility to force a certain type of reply length.
Another, less exact way would be to use something like max-tokens and always bill for that amount. This would require some playing around with the prompts but is doable in general.
https://platform.openai.com/tokenizer
You can use OpenAI official tokenizer library to calculate the actual tokens used and calculate the price according to the price of various models for completion APIs.
For Assistants API, it seems not very clear so far how to calculate the actual costs, especially for run steps.
Indeed, it seems a mess. For instance, I am not using GPT 4 as so the costs will be low, but it is using GPT 4, guess, for generating images. They charge something like: Assistant + model + extras.
This is the challenging part: the under the hood. It is not clear for instance what they are charging under the name “Assistant”.
I absolutely agree.
It’s a solution with potential to the upside.
Regarding your specific question I think the general issues remain. It’s not possible to get an exact price in advanced.
You will likely make faster progress if your customers pay in advance and you deduct the costs from their prepaid credits, for now.
did you get an answer to this? most of the comments below refer to estimating in ADVANCE which is not what you asked.
is there an openAI API call that gives us realtime billing info? Then in theory you can call it before and after an API call to estimate costs for a certain type of call. Then given token counts, you can use that guide in future, esp when you have lots of concurrent requests and don’t want to do a billing call on every single API call.
Another way maybe if you could use an API call to generate API keys per customer, and then use that for billing?
Sadly, not yet. I am still not sure how to do it
In fact, I have found a solution, at least, for chatGPT 3.5 Turbo. You can get it from data.usage.total_tokens
data is the response back from openAI API.
const output = await fetch(url, options);
const data = await output.json();
I am using Velo (Wix), which is JavaScript based.
Perhaps the challenge will be on you to come up with a number that can go into your app UX, and increase over a period of time during the app run.
An easy example I’m working through, that may give you some ideeas; 3 agents work together, people setup the agents, they hit run, if I set the tokens to be 300 for two agents, and 1000 for a manager agent, set them to meet X amount of times, I found it’s exactly 1 penny each time. This was really helpful with understanding what could happen at 1.6k tokens * 2 meetings.
In the UX, I will add a calculator, it will total the tokens and show the costs based on this equation above. As I progress the app further I can better understand the costs and ultimately update things based on this interaction.
If you have run the app 10 times, average the costs per call, divide it by minutes, and then seconds. Then I’d show that number as a growing value if that’s the desire. However you may not want people seeing this running aggregator, and this is something I learned that not all products need this face value number, perhaps hiding it will fit better, just throwing it all out there.
Really, best of luck to you. Sounds like an inspiring app. 