I’m doing a categorization exercise on thousands of customer feedback. My prompt is long where I ask it to categorize my feedback into specific categories and I provide examples of each category. I end the prompt with 1 customer feedback and make thousands of such requests (Each call carrying the exact same prompt with just the feedback changed at the end).
Is there a better way to do this where I can send the instructions first and then just send the feedback in subsequent calls? (Using gpt-3.5)
Have you tried sending batches? After instructions give it one user/assistant pair (or not) then give a list of the reviews and ask to classify each. Might not be as accurate, but I’d give it a try.
Or you might want to vectorize the categories (from your instructions) and then just vectorize the new feedback and match against vectors of your categories for closest match. If none are close enough, send that one through on its own?
Yes, there is a better and cheaper way: Embeddings.
There’s a couple of different options but the one I tend to find most intuitive involves creating a database with embeddings of different customer feedbacks that are reflective of your different categories. That is, you create embeddings of the actual feedback and then add the category as metadata. Depending on how “heterogeneous” your feedback messages are, you want to have a several examples for each category to reflect that diversity.
Once you have created this database, you can then proceed to perform the classification with the full set of feedback messages. You achieve this by converting a feedback message into an embedding as well and then performing a search against the database using cosine similarity as distance metric to get the top matches. The category (stored as metadata) associated with the match is the classification of the feedback message.
Once you have completed the preparation, the actual classification will likely just take a few minutes and cost well below one dollar.
Thanks for the responses. I gave clustering just as an example - my real question was about how to avoid sending the same prompt multiple times (if the prompt is long but the data it acts on is small).
Another example would be: Is this customer review asking for a new feature to be added? (True/False response). My prompt will be really long - but the review itself is a short sentence. Today, I end up sending the same prompt 1000 times for 1000 customer reviews. How do I just send the prompt once, and then send the review only 1000 times.
You can format your instruction/system prompt to support multiple inputs. So in the one user message you send, there are already 1000 customer reviews (taking into consideration input token limit). Then you should receive replies for each in the assistant reply.
This way, you only sent the instruction once to process X number of customer reviews.
I have tried that approach - sending the prompt with hundreds of reviews at once - however, after the first hundred or so, it starts giving erroneous results, almost as if it has forgotten what it was supposed to do. Im looking for a way to send an instruction that stays in context