How to track individual usage?

Hey! I want to track the usage of users on my platform which is powered by GPT3.

I basically want to measure how many calls a particular user does so that I can price them according to their usage itself. Is there any particular method in achieving that?

Thank you!

6 Likes

It depends on the platform you are using. @DutytoDevelop may know more.

1 Like

Hey, thank you for the reply. Well, I wanted to ask if there is a possible method where I can keep a track of them using Open AI itself?

Thanks for the mention @sps. There is a way to pull up usage of OpenAI itself.

@dhruvbhardwaj3012, can you be more specific as to what metrics you’re wanting to track? Are you looking to grab data similar to the data provided by OpenAI through their usage page?

1 Like

Hey Nicholas!
My clients now will include Writing Agencies and organizations , hence I want to have enterprises plans for them.

I am thinking of having a ‘Pay As You Go’ plan which shall be related to the usage of that particular agency. In that regard, if there is a possible method where I can track how many calls a particular client of mine has done, then I can easily charge them accordingly.

Also, yes some like the Usage Page by OpenAI would work.

Thank you for your reply!

2 Likes

Edit [3/14 2:06 AM]: Made important corrections and also changed the code to make the start_date and end_data parameters dynamic and grab today and tomorrow’s date relative to when you run the code. The code should work now!

I wrote a quick script in PowerShell to grab each user and their usage. Since I only have a personal organization because I’m a lone developer, I do not know if this will work for grabbing each user within an organization. For best results, assuming you’re the owner of the organization, I would use your API key for the following script and see if you’re able to access the user usage data within the organization or not. Also, if you need the script written in Python, I could also get that written up for you as well.

Please perform the following steps to (hopefully) get the usage data for each user for only this day within your organization:

  1. Go to OpenAI organization settings page and note your organization ID and API key
  2. Code to grab all users and their usage data in your organization (Need your organization ID):
$openai_org_id = "<org-id>"
$openai_api_key = "<api_key>"
$today = Get-Date
$tomorrow = Get-Date $today.AddDays(1) -Format "yyyy-MM-dd"
$today = Get-Date $today -Format "yyyy-MM-dd"
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$users = Invoke-WebRequest -UseBasicParsing -Uri "https://api.openai.com/v1/organizations/$openai_org_id/users" `
-WebSession $session `
-Headers @{
"method"="GET"
  "authority"="api.openai.com"
  "scheme"="https"
  "path"="/v1/organizations/$openai_org_id/users"
  "authorization"="Bearer $openai_api_key"
}
$your_users = @()
foreach ($line in ($users.Content | Out-String | ConvertFrom-JSON)) {
      $your_users += $line.members.data.user | Select-Object id,name,email
}
$each_user_usage = @{}
foreach ($user in $your_users){
	$id_of_user = $user.id
	$data = Invoke-WebRequest -UseBasicParsing -Uri "https://api.openai.com/v1/usage?end_date=$tomorrow&start_date=$today&user_public_id=$id_of_user" `
	-WebSession $session `
	-Headers @{
		"method"="GET"
		"authority"="api.openai.com"
		"authorization"="Bearer $openai_api_key"
		"openai-organization"=$openai_org_id
	}
	
	foreach ($line in ($data.Content | Out-String | ConvertFrom-JSON)) {
      $each_user_usage[$user.email] += @($user.name,$user.id,$line)
	}
}
/* 
$each_user_usage["<user's email>"] ->  (user's name, user's id, user's usage data)
$each_user_usage["<user's email>"][0] -> user's name
$each_user_usage["<user's email>"][1] -> user's id
$each_user_usage["<user's email>"][2] -> user's usage data
...
*/
3 Likes

Thanks @DutytoDevelop this is very thorough. TBH I never knew this existed. Is this documented?

1 Like

@sps,

Thank you! This is not documented, however I was able to use Google Chrome Developer Tools to be able to find the API endpoints linked to pulling OpenAI users based on the organization ID provided (whenever your API key has appropriate user access for that data) and then looked at how each user’s usage data was pulled too which seemed to be with their user ID coupled with organization ID and just linked it together in stages!

1 Like

Woah! I’ve got a lot to learn from you :raised_hands:

1 Like

@sps,

I try! I’ve had to put a couple projects on hold for the time being but I’ve started to be more active on here recently so maybe I can make a post on a couple more neat API endpoints that could benefit people by encorporating more data into their existing projects!

Btw, the API usage endpoint specifically can have a time window spanning no more than 3 months I believe or else an error gets thrown.

1 Like

Yes @DutytoDevelop, you should definitely write on it. I can’t wait to read your post.

1 Like

If I understand @dhruvbhardwaj3012’s question correctly, they want to track usage for the end users of their product, whereas the solution posted above pulls usage for OpenAI users who are members of your organization.

While we do allow you to pass a end user ID along with your requests, we do not currently support querying usage by these end user identifiers. For now you would have to build a custom system that tracks in your own database how many requests / tokens each of your customers is using.

4 Likes

Ah, thanks for the clarification! I was under the impression that end users that signed up for a service that uses GPT-3 would be added into the service owner’s OpenAI organization.

Thanks @dschnurr!

1 Like

Ooh, yeah. Something like this would be cool if it pulled from the USERID you can now send to the API?

2 Likes

Exactly David! We understood it as well. Also, cant thank @DutytoDevelop for his help. Thank you for the detailed answer. Glad to learn from you

1 Like