Hi all! I am working on a web platform for you to see LLM API analytics on a per-user basis and set user limits to avoid usage spamming attempts.
I have also started developing a ML-based classification model to try and detect prompt injection inputs – Of course this is not a good solution now, but I am curious (hence the username) to learn as much as I can and try out new solutions and iterate. Prompt injection seems to be troubling even the likes of Microsoft, and so even rudimentary solutions may be of value to resource-limited app-developers like myself!
I would love to open a thread about what problems LLM app-devs experience and what tools may be helpful. Do you guys think the biggest issue is model finetuning, security, API cost or something else? I think the whole field is thinking of amazing applications for LLMs, at an unprecedented rate, but the framework for a robust and safe deployment is severely lacking!
What tools do you use to make develop/deploy your LLM apps?
Here are some screenshots, I have set up a website llmetrics.app to display some features I am working on. Feel free to take a look!
