Dude, thank you! This stuck in my mind. The issue I saw was depending upon end users to rate the responses.
So today I went into the playground and came up with a prompt that can read the recorded questions and answers and render a rating itself based upon it’s best judgement. Now I can regularly rate all responses and over time come up with a list of high quality responses that can be used for fine-tuning or embedded themselves for greater accuracy.
@SomebodySysop have you considered providing prepared prompts to your users for common tasks? Please let me know if it makes sense. I am working on a process for this.
No. Although I am thinking about working with a publisher of Real Estate exam study guides to pre-populate the dataset with questions AND answers, which would be similar to the same concept.
How many questions do you estimate there will be? I’m guessing you are adding information to a database and will use embedding to pull relevant data. That is not necessary if you have well crafted prompts.
From end-users? If we are successful at marketing this to everyone who could use it, potentially thousands.
Users are entering the prompts. They are asking questions which hopefully are answered in the database. Which is why I feel it is necessary to at least try and give them some basics on prompt engineering.
you can fine tune a model with most of the prompting predone so that they could use their natural language to type to it… you would need to know the stuff they are going to ask to keep it more pinned to task… otherwise … call a meeting get everyone in there and really explain it to them as if this is their job now
My advice is to ask a few of your potential customers to come into a virtual room for an hour of free access to your service. Record their use, categorize each of their interactions (that did what they actually wanted it to do), then build a UI layer that does only those things. Later customers will come to you asking for more functionality, but keep it simple at first. Good luck! Can’t wait to try what you built.
Now I understand. This AI tech is going to have a big, and immediate impact on a lot of job markets. Whereas in the past, learning how to use boolean operators in search was a good skill to have, EVERYBODY is going to have to get good at prompting skills just to keep their existing jobs. Yes, an exaggeration, but you know what I mean.
It feels bigger than the internet. But we should expect a bubble, heh?
Like I previously suggested, my bet is that the average user won’t have to prompt a lot. It’s a great skill but likely can be replaced by a minority that creates the prompts for others to implement and the customers to use.
