It would be very helpful to check the hard usage limit for an account associated with an API key & the overall usage.
This would allow ensuring the maximum liability for any errors in the calling code. It would definitely make things easier for legal/accounting purposes.
I’m not sure if you realize this, but on every completion api call, the OpenAI models return this information:
Array
(
[id] => chatcmpl-7BJmdfVh4Y7RqfzvglSud8qBNomnk
[object] => chat.completion
[created] => 1682932011
[model] => gpt-4-0314
[usage] => Array
(
[prompt_tokens] => 1216
[completion_tokens] => 33
[total_tokens] => 1249
)
[choices] => Array
(
[0] => Array
(
[message] => Array
(
[role] => assistant
[content] => The "City on the Edge of Forever" episode of the original "Star Trek" series, written by Harlan Ellison, first aired in April 1967.
)
[finish_reason] => stop
[index] => 0
)
)
)
So, in my code, I get the total tokens for each call and add them up.
| // No error occured, get usage info. | ||
|---|---|---|
| $id = $responseBody[‘id’]; | ||
| $usage = $responseBody[‘usage’]; | ||
| $prompt_tokens = $usage[‘prompt_tokens’]; | ||
| $total_tokens = $usage[‘total_tokens’]; |
You will have to create a way to store this tagged by user, but that should be relatively simple. Now you have a log of tokens expended per call by user that you can check with a simple sql command to make sure the user is under the token limit you have set.
That’s definitely the main approach, but we were hoping for multiple levels of redundancy. We’re all human, assuming that a company’s code for tracking usage goes wrong, or even that it’s a no-code solution that incorporates OpenAI and is misconfigured by the user, it could lead to a situation where hundreds if not thousands of dollars are racked up before someone notices.
It would be really nice just to have that additional level of security where we can leverage OpenAi’s existing usage tracking. And that’s the other thing, this feature is there, it’s hard to justify the development cost if a couple days from now they suddenly make it available.
I found this: How can i check OpenAI usage with Python? - #5 by weibing.chen.xy
Can’t find any documentation on it, but I converted the code to PHP and ran it and darned if it didn’t work!
This is the data it comes back with (I ran it with date=2023-05-01):
Array
(
[object] => list
[data] => Array
(
[0] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 3
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 3
[n_context_tokens_total] => 322
[n_generated] => 3
[n_generated_tokens_total] => 53
)
[1] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 229
[n_generated] => 1
[n_generated_tokens_total] => 22
)
[2] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 2
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 2
[n_context_tokens_total] => 21
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[3] => Array
(
[aggregation_timestamp] => 1682923800
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 239
[n_generated] => 1
[n_generated_tokens_total] => 25
)
[4] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 7
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 7
[n_context_tokens_total] => 1217
[n_generated] => 7
[n_generated_tokens_total] => 227
)
[5] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 4
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 4
[n_context_tokens_total] => 5576
[n_generated] => 4
[n_generated_tokens_total] => 176
)
[6] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 4
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 4
[n_context_tokens_total] => 48
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[7] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 1
[n_context_tokens_total] => 99
[n_generated] => 1
[n_generated_tokens_total] => 9
)
[8] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 1193
[n_generated] => 1
[n_generated_tokens_total] => 81
)
[9] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 1
[n_context_tokens_total] => 4
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[10] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 8
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 8
[n_context_tokens_total] => 1714
[n_generated] => 8
[n_generated_tokens_total] => 256
)
[11] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 4
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 4
[n_context_tokens_total] => 5231
[n_generated] => 4
[n_generated_tokens_total] => 184
)
[12] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 4
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 4
[n_context_tokens_total] => 74
[n_generated] => 0
[n_generated_tokens_total] => 0
)
)
[ft_data] => Array
(
)
[dalle_api_data] => Array
(
)
[whisper_api_data] => Array
(
)
[current_usage_usd] => 0
)
Yes, i need this to show on my app, telling the user about their current balance.
What I actually ended up doing, instead of querying OpenAI after the fact, I count the tokens in the responses returned during my API calls. I record them into a SQL database, keyed by user, and they can see their balance in their account at any time.