Hey! I want to track the usage of users on my platform which is powered by GPT3.
I basically want to measure how many calls a particular user does so that I can price them according to their usage itself. Is there any particular method in achieving that?
Thank you!
It depends on the platform you are using. @DutytoDevelop may know more.
Hey, thank you for the reply. Well, I wanted to ask if there is a possible method where I can keep a track of them using Open AI itself?
Thanks for the mention @sps. There is a way to pull up usage of OpenAI itself.
@dhruvbhardwaj3012, can you be more specific as to what metrics you’re wanting to track? Are you looking to grab data similar to the data provided by OpenAI through their usage page?
Hey Nicholas!
My clients now will include Writing Agencies and organizations , hence I want to have enterprises plans for them.
I am thinking of having a ‘Pay As You Go’ plan which shall be related to the usage of that particular agency. In that regard, if there is a possible method where I can track how many calls a particular client of mine has done, then I can easily charge them accordingly.
Also, yes some like the Usage Page by OpenAI would work.
Thank you for your reply!
Edit [3/14 2:06 AM]: Made important corrections and also changed the code to make the start_date and end_data parameters dynamic and grab today and tomorrow’s date relative to when you run the code. The code should work now!
I wrote a quick script in PowerShell to grab each user and their usage. Since I only have a personal organization because I’m a lone developer, I do not know if this will work for grabbing each user within an organization. For best results, assuming you’re the owner of the organization, I would use your API key for the following script and see if you’re able to access the user usage data within the organization or not. Also, if you need the script written in Python, I could also get that written up for you as well.
Please perform the following steps to (hopefully) get the usage data for each user for only this day within your organization:
$openai_org_id = "<org-id>"
$openai_api_key = "<api_key>"
$today = Get-Date
$tomorrow = Get-Date $today.AddDays(1) -Format "yyyy-MM-dd"
$today = Get-Date $today -Format "yyyy-MM-dd"
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$users = Invoke-WebRequest -UseBasicParsing -Uri "https://api.openai.com/v1/organizations/$openai_org_id/users" `
-WebSession $session `
-Headers @{
"method"="GET"
"authority"="api.openai.com"
"scheme"="https"
"path"="/v1/organizations/$openai_org_id/users"
"authorization"="Bearer $openai_api_key"
}
$your_users = @()
foreach ($line in ($users.Content | Out-String | ConvertFrom-JSON)) {
$your_users += $line.members.data.user | Select-Object id,name,email
}
$each_user_usage = @{}
foreach ($user in $your_users){
$id_of_user = $user.id
$data = Invoke-WebRequest -UseBasicParsing -Uri "https://api.openai.com/v1/usage?end_date=$tomorrow&start_date=$today&user_public_id=$id_of_user" `
-WebSession $session `
-Headers @{
"method"="GET"
"authority"="api.openai.com"
"authorization"="Bearer $openai_api_key"
"openai-organization"=$openai_org_id
}
foreach ($line in ($data.Content | Out-String | ConvertFrom-JSON)) {
$each_user_usage[$user.email] += @($user.name,$user.id,$line)
}
}
/*
$each_user_usage["<user's email>"] -> (user's name, user's id, user's usage data)
$each_user_usage["<user's email>"][0] -> user's name
$each_user_usage["<user's email>"][1] -> user's id
$each_user_usage["<user's email>"][2] -> user's usage data
...
*/
Thanks @DutytoDevelop this is very thorough. TBH I never knew this existed. Is this documented?
@sps,
Thank you! This is not documented, however I was able to use Google Chrome Developer Tools to be able to find the API endpoints linked to pulling OpenAI users based on the organization ID provided (whenever your API key has appropriate user access for that data) and then looked at how each user’s usage data was pulled too which seemed to be with their user ID coupled with organization ID and just linked it together in stages!
Woah! I’ve got a lot to learn from you 
@sps,
I try! I’ve had to put a couple projects on hold for the time being but I’ve started to be more active on here recently so maybe I can make a post on a couple more neat API endpoints that could benefit people by encorporating more data into their existing projects!
Btw, the API usage endpoint specifically can have a time window spanning no more than 3 months I believe or else an error gets thrown.
Yes @DutytoDevelop, you should definitely write on it. I can’t wait to read your post.
If I understand @dhruvbhardwaj3012’s question correctly, they want to track usage for the end users of their product, whereas the solution posted above pulls usage for OpenAI users who are members of your organization.
While we do allow you to pass a end user ID along with your requests, we do not currently support querying usage by these end user identifiers. For now you would have to build a custom system that tracks in your own database how many requests / tokens each of your customers is using.
Ah, thanks for the clarification! I was under the impression that end users that signed up for a service that uses GPT-3 would be added into the service owner’s OpenAI organization.
Thanks @dschnurr!
Ooh, yeah. Something like this would be cool if it pulled from the USERID you can now send to the API?
Exactly David! We understood it as well. Also, cant thank @DutytoDevelop for his help. Thank you for the detailed answer. Glad to learn from you
Is support for this on the roadmap?
Hi Nicholas @DutytoDevelop thanks for the code!
I just tried to run it and there was this error message:
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest)
[Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.I
nvokeWebRequestCommand
Invoke-WebRequest : {
“error”: {
“message”: “Missing query parameter ‘date’”,
“type”: “invalid_request_error”,
“param”: null,
“code”: null
}
}
Is this because the API of OpenAI has been updated since then? Could you please tell me how to fix this? Thanks!
I am also interested in a similar feature for tracking user token utilization, in a way that user’s pay for their own tokens directly, without having to create an OpenAI account. For example, child elements with their own secret key, tracking their own usage, and each one can have their own payment method tied directly to their key. However, rather than having a “reader” access, I wouldn’t want them to be able to see organizational data such as who the other customer’s are. Some sort of bridge between the customer model and the organizational model. Having a more simple route of abstraction would do wonders.
This is amazing, thanks for sharing this. Would you be willing to share a python version of this code?
Well, you will need to create these users in your DB, right? So why not add token usage to your DB as well - you will know who is using OpenAI and with response you will receive number of tokens, which you will save into your DB and connect with the user that used it. That way, you could show them exactly what they are paying for.
