There are a lot of conversations about “pay-as-you-go” in this developer forum, but they all relate to using OpenAI specifically. But, what about us developers who create systems using the OpenAI API that we “re-sell” (I guess that’s the word) to our end-users?
Using the plethora of “Chat with your PDF” services as an example, most seem to be using gpt-3.5 and charging a set monthly amount, based on anticipated usage.
So, to me, the simplest, easiest, least painful way to go is implementing pay-as-you-go for the end-user. My system can do all the necessary calculations monthly: I can hard code it, or use my shiny new Text to SQL implementation.
My question here is sort of open ended: I’m curious if anyone out there has been doing this, and what have been the issues? And, of course, just general thoughts on this as approach as well.
I mean, billing for tokens used makes perfect sense to me and you – but I’ve been working with OpenAI models closely for about 10 months now. What about the average Joe who has never head of a “token” before?
Anyway, thoughts? Anyone?