Do you use any service to help you manage your prompts? We are creating a QA service and some of us are trying different prompts and parameters to find the best option. But keeping track of changes and providing feedback etc is hard to do. I feel that a service should exist to keep track of different versions, provide feedback, and auto-experiment (ie. given a base prompt, launch it with different parameters to see how the changes affects the result - or even automatically iterate on the prompt itself)
Potentially, the service could even analyze the prompt and give suggestions on how to improve it depending on the objective.
Does something like this already exists? How do you manage this workflow?
I don’t know of anything that does that off-hand, but it sounds like a cool idea. I’m sure you might be able to use an off-the-shelf CMS like Drupal or WP even to set it up, although something custom would probably work better.
Maybe make it so people entering prompts could share with the public or keep it private…
Come to think of it, I believe there’s a prompt market or two? I’ve never browsed or bought a prompt, though. Too much fun trying to come up with them on my own.
Thanks @PaulBellow ! The “market-ish” part could also be an option, but I’m thinking more on a private use for now: help teams to iterate, refine and keep track of prompts; also help testing when a new version of the model comes out, be able to test against different APIs… On top of these several things could be built.
Your problem is actually very common and that’s why we are building a prompt management system! We are currently in the process of onboarding beta users, you can sign up to our waitlist here: https://usegenie.app/
It is currently hard to manage and provide feedback on the prompts, despite the fact that it can be crucial for the product! We’re looking to change that.
Please don’t hesitate to contact us with any questions or feedback you may have. We look forward to hearing from you!