I am not using a real api key. this is just for the demo
1 Like
same requirement here, would be great to have it right in the dashboard
1 Like
did you ever find a solution for this @bzhang? if not we have one
I second this idea, this is very important for me because I have multiple products running on OpenAI services. If we can not measure by API key, can we apply a business code and we can categorize usage by business code?
If you have as many mobile phones as you have products, another tracking solution that is currently available is members of an organization, where you invite an email address.
Here is a link to a script that shows the endpoint that can be used to track organization user usage: https://github.com/dav-ell/toktrack.
(edit, this function of accessing undocumented APIs by your API key has been removed)
I won’t post the topic it came from because it has a dangerous website linked - (which can be described as “put in your API key for us to steal, and we’ll block showing our own abuse in our tracking until it’s too late”)
It is ridiculous and anti-enterprise to require unmanaged accounts that are validated not against your organization but by a phone number and their own credit card; that each phone number can only be used twice and account deletion renders the phone number unusable. The whole system should be trashed and re-thought.
1 Like
Oh yes! Inviting another e-mail is okay? How does it work actually?
It can actually be described as “inviting someone to make their own OpenAI account if they don’t have one” - which will then be a member of your organization, and can choose if they want their own API keys billed to you by default or per-API-call.
The actual per-user tracking UI from an administrator standpoint is still quite subpar. The script demonstrates use of an undocumented API (and an API change in May broke another access method that was reverse-engineered).
The only way it is reasonable is to be the owner of all accounts - which is practically lost the second you let an soon-disgruntled employee use the secondary account. Maybe only as a mitigation to see which of your apps has a compromised API key.
The benefit is that you don’t have to put credit card numbers into secondary accounts in order for others to bill and use (unlike ChatGPT plus, another company billing cluster**)
I’ve tried doing this request numerous times but end up with the following error:
Your request to GET /v1/dashboard/billing/usage must be made with a session key (that is, it can only be made from the browser). You made it with the following key type: secret.
Anyone know how to get around this?
1 Like
I don’t presume to read any better than the next guy, but I think that instead of using your API key, which has been disabled as an authentication method, you’d have to use the https TLS session key, that is used for handshake and then encryption when you authenticate with your login on the openai website to manage your account - (and then look under usage).
Can’t access the TLS session key. It seems like the TLS session key is meant for post-logging into the OpenAI portal which is not my intention as I’m looking to make the request from a web app. I’m also confused why the Postman request didn’t work since I have the exact same parameters as the initial post (other than a different API key).
I solved this like so:
- In our API the requests to do a call to OpenAI’s api include the userID that is requesting it.
- We spin a process with the request to OpenAI and we keep the parameter of the UserId associated to this process
- When the api call returns, we capture the number of tokens used. This is part of the json returned by the completion api endpoint.
- Now we have: the userId and the tokens. We just save that in a database.
At the end of each cycle (daily, weekly, etc) we know exactly:
- Who consumed how many tokens, which prompt was used, and even the response.
Thoughts?
3 Likes
We use the streamed responses and for whatever reason those do not include the token cost.
I can’t really get my head around why the cost per api key is not available.
Even if it was only presented once invoiced, that would help.
You should keep track on how many tokens are send when a process has an error e.g. timeout. It’s also charged. I suggest to mark them as faulty.
Yes, please add the feature. Need to track spending
2 Likes
are you searching for that?
No, I want to see spendings for each API key separately
You will have to implement that by yourself.
By counting tokens.
There is tiktoken to count the number of tokens send by your api request (that is also charged even when the request times out or when you send to much for the model).
And there is a usage object in the response where you have stats.
And then you implement that into your application.
In case you have a team of developers who all have their own key i think you don’t have any chance to reliably count it.
1 Like
And really, the usage tracking should not be per API key at all. The number of keys that can be generated is limited, and key values cannot be recalled within the UI.
Instead, one should implement their own user account system. You can thus track individual usage for hundreds, or even policy violations.
The individual user ID can be reported as a field during the API call, which was so that OpenAI can see you have users violating policy, and it is not you yourself.
Just one API key is needed then, stored more securely as an environment variable. The API key can be regularly and easily changed or disabled in event of leak and abuse.
1 Like
This is a reasonable request. What happens if someone loses an API key and it starts being used extensively, or some program has an error and is making more calls than desired. It’s reasonable to want to remove that key, but we don’t really have a sense for which key to remove do we?
2 Likes
Why don’t you build an API gateway to connect/proxy all applications and devs on and get rid of multiple keys?
I mean it has many advantages other than just control in case of stolen keys. You also have a centralized log, retry logic and you can also see which application / developer does what.
And how long would it take to build? A day?
Plus: I have build a queue to never have to worry about max tokens per minute.
I simply don’t send more requests per minute.
I need another job done on that API? It runs through my API Gateway.
Centralizing the API connection does make alot of sense.
And it also has another advantage. I have set the connections to openai into a standardized class with an interface. And I have made the same with my deployed models on azure.
And there is a strategy pattern upfront that redirects the requests to different models.
Just in case anyone wonders why you should use depency injection.
At least when I am not mistaken in thinking TPM counts per account and not per key, right?
2 Likes