Hi all! I have noticed that several organisations and LLM-app developers find it difficult and time-consuming to create gateways to monitor and control user usage.
To solve this, I am helping develop a (hosted) LLM-specific API gateway (https://llmetrics.app) with an admin dashboard where you are able to track user-level costs accurately and robustly set cost limits and access rules. Behind the scenes, the server creates new derived API keys for each user, and handles user authentication and authorisation. With this abstraction, you can just create user groups (and add users to them) with custom cost and usage limits.
Please feel free to shoot me a DM or sign up (on the site) if these issues resonate with you or your organisation on your LLMOps journey, the product is in Beta but we have been giving out free trials for testing!
Hi all, we are building an LLM API gateway for businesses to be able to easily track and control usage and cost (on a user-level). It (LLMetrics) includes authorisation/authentication, an admin dashboard for monitoring and control (user/group cost limits) and will soon include load balancing between multiple LLM providers.
If anyone in this forum is experiencing difficulties in budget and access management for LLM APIs, please reach out and we can supply a free trial for the Early Access (llmetrics.app)!
1 Like
Hi wanted to ask how far have you gotten along with this project? Curious if you’ve made it through to an alpha version
Hey!
We do not have an alpha version yet unfortunately, we have a limited feature Beta version which we are slowly giving out and testing with businesses. A lot of time has been spent on problem and customer discovery,
Please have a look at the website (https://llmetrics.app) and let me know if you are interested in receieving an Early Access account 
Updated original screenshots and text in main post to reflect current state of the product
Hi all,
I am wondering whether OpenAI is working towards supporting cost management for consumer-facing applications with many users. With LLM API calls becoming more and more complex, it’s quite hard to accurately keep track of and control user costs in deployment. As well, it would be cool to generate billing and usage reports for each user to pass down LLM API costs and optimise pricing strategies. I am working on a solution to this (https://llmetrics.app, hosted cost management platform for LLM APIs) but wondering OpenAI’s approach to this, as well as other members of this forum. Please let me know how you solve this issue
