My company will use different API key for different AI-enabled applications/features/engineering teams. We’d like to track usage and cost by API key. Is that a way to track API usage and cost by API keys?
It seems right now the usage tracking is by organization only.
It would be fine if we need to track it as middleware when hitting OpenAI API. Just want to make sure I am not missing a hidden feature.
What we do is log a token identifier, and the response model name and tokens used, for each API request, into our log/metrics/tracing/observability system.
We then sum this up in a worksheet with a simple query.
Observe Inc has a simple OpenAI integration: What's New: OpenAI App!
However, you’ll still need to add instrumentation to your own code to log out the model/token counts, and write a query to parse that back in and sum it up.
(Observe is an enterprise software suite; there’s no free trial or open source version; if your volumes are low, doing the summary in Python or SQL or somesuch might be tractable as an alternative.)
Just finished doing the same exact thing, using SQL. OpenAI sends you the token usage on every query, so I just record it, along with user, model, date, IP address and a couple other parameters. This best and easiest way to do it, IMHO.
+1 for tokens/cost breakdown per API key (which sums up to the total).
Especially important for chat streaming responses which don’t include a token count, and puts the burden to all downstream apps to re-tokenize the responses and maintain databases for what’s already an upstream state.