We are in the process of creating a product that utilizes the OpenAI API. This product will be offered to end customers who will use it. Our goal is to keep track of how our product is being used, and we intend to charge end customers based on their usage of the product. To achieve this, we are looking for a solution to effectively monitor and bill for the usage of our product integrated with the OpenAI API. If you have any recommendations or insights on how we can implement this, please feel free to share your thoughts.
However, when streaming - displaying words as they are generated and how you’d want many customer-facing products to operate - you don’t receive that token count.
You can calculate tokens exactly with an encoding library, tiktoken.
Then you can determine if it is more practical to charge by your exact billing, or whether instead to present per-query costs, per character, per month with quota and purchase of boosters, etc.
Yes API sends input and ouput tokens that pricning will be charged by Open AI for the tokens which i consumed but here i want sytem /application where i need to monitor the input/output token consumed by my cusomter through our product