It looks like the most common way to charge users’ on sites that use AI APIs is either to implement your own credit system, which I think a lot of sites are doing these days and each have their own way of measuring these, or I could just charge a premium on the actual cost of using AI (e.g. Cursor does this in some situations).
And in my situation, because of the specific nature of my product, I would like to do the latter, but I have to implement the system of billing and metering myself.
Writing the code to track this manually is certainly doable, but it seems like either something that OpenAI should provide (I had higher hopes for the Costs API) or there should be some open source solution.
In an ideal world the Costs API would work such that I could break it down by metadata that I send per request, so that I could then tag requests with e.g. a user ID and then I could query for the cost grouped by metadata tags.
Does anyone have any insight on this topic?