Counting tokens yourself is not very reliable. In the past, I used OpenAI’s recommended tool to count them (gpt3-tokenizer) and found some minor offsets in what I counted and what I was billed. I believe I saw a line somewhere on their website saying that OpenAI may count more tokens for some other misc processing… I’ll edit my response if I find it again.
However, as thomas pointed out, this adds complexity and has it’s own load of burden. How can the user trust my count? How can you detect miscalculated counts? How do you handle overcharges/undercharges?
At the very least, there should be clear documentation in OpenAI’s usage policies, terms of service and API reference on the matter. I actually didn’t find any clear line in any policies which would suggest that it would be a violation for a third-party application to use other users’ API keys on their behalf. Maybe it’s worded in some lawyerly way I missed… I stumbled on multiple BYOK posts with replies from staff that still weren’t clear, pointing to links that no longer exist or talking about an “app review process” that was abolished in favor of a manual, per-case basis.
Nevertheless, I shouldn’t be finding such sensitive policies in a forum. I want to develop with OpenAI’s products but don’t want to risk losing my access to the platform because I wasn’t able to find a clear “green light”.