Hey All,
Open Source Project
I made a plugin a while back early on in the OpenAI beta launch. I never submitted it as a plugin though. It was mostly just a way for me to experiment and investigate OpenAI’s methods. However, recently, I needed to test out new tools I am making and I needed an installable plugin with a grassroots population using it. You can find the plugin under “ProdigiLink Video” in the store and the source code can be found at Codie-Petersen/YoutubeSearchPlugin.
Metrics Logging Service
The other tool I am developing is for plugin metrics (technically you could use it for anything to be honest not just plugins). It is called “ProdigiMetrix” and you can find the source/python package at Automacene/ProdigiMetrix-Python. I’ll be writing up the documentation for it right now including the installation method. However, there is no signup page for it as of yet. I can manually add your plugin to the database if you want to start logging metrics like keywords and times. The current version is free because I don’t know if it is something that will be worthwhile to developers. I suspect the two supported versions are simple enough they will stay free even if I end up monetizing the service.
Account Access and Examples
The YouTube search plugin is currently using it. You can check out how to use it and if it would be something you’d find useful. It is in the early stages but if there is interest, I will be adding extras like custom metrics, user tracking, etc. I don’t think the infrastructure costs are too much, so I’m pretty sure even a paid tier will be pretty cheap.
If you have any questions just go ahead and ask them here and I’ll answer them. If you want an account just let me know and I’ll send you a plugin id and authorization token in a DM which are required for logging metrics. Like I said, this is an experiment, so if you think something would be cool to have and you have a plugin, just let me know and we’ll see if it something that can be added.
Concerns
Here are some common questions and concerns I have had from people. I imagine others will have similar thoughts. So, I’ll just mention them here to avoid long unrelated thread derails.
Metrics: Yes they are needed. What gets measured, gets improved. If you don’t know your audience, then you’ll never be able to cater to their needs/wants/requests.
Privacy: In its current form, all keywords are obfuscated and anonymized by proxy of how simple the metrics are. In the backend, I don’t even have time labels matched to the keywords. You can only track keywords (and their frequency, possibly you can infer language and region from the keywords), timestamps, and how many times prompts have been made. There is no user data right now, and the metrics are all separate from each other.
X can already do Y: This project is a pinky toe in the water compared to the full dive I want to make. If basic metrics like these are all you want, then I would suggest using other services because as/if my project grows, it may bloat your deployment.
You have ads in the YouTube plugin: You can delete them if you want, I will not be. It’s an opensource example.
There are already video summary plugins: Correct. It’s an example, portfolio project, and sandbox. You don’t have to deploy it. It just exists for anyone curious. I’ve only deployed it as an “in the wild” experiment.
Javascript library: I only have python right now, but you can take a look at the ProdigiMetrix-Python project. It is pretty much just a hardcoded wrapper around RESTful API requests. You can implement your own if I am moving too slow. BUT… I’ll get on it ASAP if there is interest from people with existing launched plugins.
Other Languages: I know C#, Go, and Java as well but I don’t think those are supported by OpenAI yet though. However like I said, there are examples in the Python package you can use to hand roll your own request code. I’ll monitor interest of other languages though.
Viewing the Metrics: There is a “get” function for both ‘basic’ and ‘keyword’ it returns a dictionary of the metrics. But I don’t have a visualizer yet. That will come after some interest.
Milestones: I’ll add those in the documentation and keep them updated.
Suggestions: Yes please. 