It would be very helpful to check the hard usage limit for an account associated with an API key & the overall usage.
This would allow ensuring the maximum liability for any errors in the calling code. It would definitely make things easier for legal/accounting purposes.
I’m not sure if you realize this, but on every completion api call, the OpenAI models return this information:
Array
(
[id] => chatcmpl-7BJmdfVh4Y7RqfzvglSud8qBNomnk
[object] => chat.completion
[created] => 1682932011
[model] => gpt-4-0314
[usage] => Array
(
[prompt_tokens] => 1216
[completion_tokens] => 33
[total_tokens] => 1249
)
[choices] => Array
(
[0] => Array
(
[message] => Array
(
[role] => assistant
[content] => The "City on the Edge of Forever" episode of the original "Star Trek" series, written by Harlan Ellison, first aired in April 1967.
)
[finish_reason] => stop
[index] => 0
)
)
)
So, in my code, I get the total tokens for each call and add them up.
| // No error occured, get usage info. | ||
|---|---|---|
| $id = $responseBody[‘id’]; | ||
| $usage = $responseBody[‘usage’]; | ||
| $prompt_tokens = $usage[‘prompt_tokens’]; | ||
| $total_tokens = $usage[‘total_tokens’]; |
You will have to create a way to store this tagged by user, but that should be relatively simple. Now you have a log of tokens expended per call by user that you can check with a simple sql command to make sure the user is under the token limit you have set.
That’s definitely the main approach, but we were hoping for multiple levels of redundancy. We’re all human, assuming that a company’s code for tracking usage goes wrong, or even that it’s a no-code solution that incorporates OpenAI and is misconfigured by the user, it could lead to a situation where hundreds if not thousands of dollars are racked up before someone notices.
It would be really nice just to have that additional level of security where we can leverage OpenAi’s existing usage tracking. And that’s the other thing, this feature is there, it’s hard to justify the development cost if a couple days from now they suddenly make it available.
I found this: How can i check OpenAI usage with Python? - #5 by weibing.chen.xy
Can’t find any documentation on it, but I converted the code to PHP and ran it and darned if it didn’t work!
This is the data it comes back with (I ran it with date=2023-05-01):
Array
(
[object] => list
[data] => Array
(
[0] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 3
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 3
[n_context_tokens_total] => 322
[n_generated] => 3
[n_generated_tokens_total] => 53
)
[1] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 229
[n_generated] => 1
[n_generated_tokens_total] => 22
)
[2] => Array
(
[aggregation_timestamp] => 1682923500
[n_requests] => 2
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 2
[n_context_tokens_total] => 21
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[3] => Array
(
[aggregation_timestamp] => 1682923800
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 239
[n_generated] => 1
[n_generated_tokens_total] => 25
)
[4] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 7
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 7
[n_context_tokens_total] => 1217
[n_generated] => 7
[n_generated_tokens_total] => 227
)
[5] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 4
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 4
[n_context_tokens_total] => 5576
[n_generated] => 4
[n_generated_tokens_total] => 176
)
[6] => Array
(
[aggregation_timestamp] => 1682931000
[n_requests] => 4
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 4
[n_context_tokens_total] => 48
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[7] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 1
[n_context_tokens_total] => 99
[n_generated] => 1
[n_generated_tokens_total] => 9
)
[8] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 1
[n_context_tokens_total] => 1193
[n_generated] => 1
[n_generated_tokens_total] => 81
)
[9] => Array
(
[aggregation_timestamp] => 1682931600
[n_requests] => 1
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 1
[n_context_tokens_total] => 4
[n_generated] => 0
[n_generated_tokens_total] => 0
)
[10] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 8
[operation] => completion
[snapshot_id] => gpt-3.5-turbo-0301
[n_context] => 8
[n_context_tokens_total] => 1714
[n_generated] => 8
[n_generated_tokens_total] => 256
)
[11] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 4
[operation] => completion
[snapshot_id] => gpt-4-0314
[n_context] => 4
[n_context_tokens_total] => 5231
[n_generated] => 4
[n_generated_tokens_total] => 184
)
[12] => Array
(
[aggregation_timestamp] => 1682931900
[n_requests] => 4
[operation] => embeddings
[snapshot_id] => text-embedding-ada-002-v2
[n_context] => 4
[n_context_tokens_total] => 74
[n_generated] => 0
[n_generated_tokens_total] => 0
)
)
[ft_data] => Array
(
)
[dalle_api_data] => Array
(
)
[whisper_api_data] => Array
(
)
[current_usage_usd] => 0
)
Yes, i need this to show on my app, telling the user about their current balance.
What I actually ended up doing, instead of querying OpenAI after the fact, I count the tokens in the responses returned during my API calls. I record them into a SQL database, keyed by user, and they can see their balance in their account at any time.
Would you mind sharing the PHP code please? (without your keys, etc.)
Thanks in advance
In this code, we return the token totals for the model API call along with the response
/**
* solrai_generateResponseOpenAI ($systemMessage, $text)
*
* Generate a response to $text using API call to
* OpenAI model $model using $systemMessage
* We can also set the $temperature
*
*/
public function solrai_generateResponseOpenAI($systemMessage, $text, $model, $temperature) {
// Define the OpenAI API endpoint and parameters
$url = $this->urlOpenAI;
// Initialize the $messages array with the system message
$message = array(
array("role" => "system", "content" => $systemMessage)
);
// Define the new user message
$userMessage = array("role" => "user", "content" => $text);
// Append the new user message to the end of the $messages array
$message[] = $userMessage;
// Define the data for the API request
$data = array(
'messages' => $message, // No need to wrap $message in an array if it is set as array above
'model' => $model,
'temperature' => $temperature
);
// Define the request headers and data
$headers = [
'Authorization: Bearer ' . $this->apiOpenAI,
'Content-Type: application/json'
];
// Call the GPT API
$responseBody = $this->solrai_executeCurlRequest($url, $headers, $data);
$apiElapsedTime = $this->getElapsedTime();
// Logging
$logger = \Drupal::logger('solrai');
# Debug
# Log the response body
# $logger->notice('(generateResponseOpenAI) Response: @response', ['@response' => print_r($responseBody, TRUE)]);
// Initialize $prompt_tokens and $total_tokens;
$id = '';
$prompt_tokens = 0;
$total_tokens = 0;
// Initialize the $generated_text variable
$generated_text = '';
// Check if the response contains an error
if (isset($responseBody['error'])) {
// Extract the error message from the response
$error_message = $responseBody['error']['message'];
// Assign the error message to $generated_text
$generated_text = "Error occurred (solrai_generateResponseOpenAI): {$error_message}";
} else {
// No error occured, get usage info.
// Check if 'usage' exists in the response body
if (isset($responseBody['usage'])) {
$usage = $responseBody['usage'];
if (isset($usage['prompt_tokens'])) {
$prompt_tokens = $usage['prompt_tokens'];
}
if (isset($usage['total_tokens'])) {
$total_tokens = $usage['total_tokens'];
}
}
// No error occurred, proceed with normal response processing
if (isset($responseBody['choices'][0]['message']['content'])) {
// No error occurred, proceed with normal response processing
$generated_text = $responseBody['choices'][0]['message']['content'];
} else {
// Set to an empty string if "choices" key doesn't exist
$generated_text = '';
}
}
// Return the generated text and prompt tokens
return [
'generated_text' => $generated_text,
'prompt_tokens' => $prompt_tokens,
'total_tokens' => $total_tokens,
'apiElapsedTime' => $apiElapsedTime
];
}
Here you see the calling method grabs the token totals from the response
Here we also add them to the running total for this sequence of API calls
$response = $this->solraiService->solrai_generateSQL($systemMessage, $sql);
# Debug
# $logger->notice('Second generateSQL call response: @response', ['@response' => print_r($response, TRUE)]);
$sql = $response['generated_text'];
$total_tokens = $total_tokens + $response['total_tokens'];
$apiElapsedTime = $apiElapsedTime + $response['apiElapsedTime'];
Finally, we write the log. Badda-bing, badda-boom. It’s done.
$logUser = $username;
$now = new \DateTime();
// Format the date and time
$logDate = $now->format('Y-m-d\TH:i:s');
$logIp = \Drupal::request()->getClientIp();
$logType = 'sql';
$logSolrId = 'n/a';
$logNid = 0;
$logModel = $modelSql;
$logTime = $apiElapsedTime;
$logTokens = $total_tokens;
$logDesc = $question;
$logSource = 'site';
if ($success === 'Y') {
$logRank = 10;
} else {
$logRank = 0;
}
$this->solraiService->solrai_logEvent($logUser, $logIp, $logDate, $logType, $logSolrId, $logNid, $logModel, $logTokens, $logDesc, $logTime, $logNotes = '', $logEmail = '', $logSource, $logUrls = '', $logRank);
Thank you Very Much @SomebodySysop
Have several nice years ahead