I have a web application that uses prompt engineering and chat completions to assist users in generating performance reports across various use cases (e.g., performance reviews, awards, etc.). The app handles around 10,000-15,000 requests per day, and I’ve configured it to store the chat completions along with associated metadata, which I can view on a dashboard. This metadata helps categorize and track how users are interacting with the application.
I’m looking for advice or shared experiences on how best to leverage this large volume of completions data. Specifically, I’m curious about two areas:
-
Fine-Tuning: Has anyone used chat completion data to fine-tune models for more accuracy and personalized results? If so, how did you approach curating and preprocessing the data for fine-tuning purposes?
-
Vectorization: I’m also exploring the idea of vectorizing this data for improved search capabilities or embedding-based tasks. For those who have gone down this path, how did you handle the data transformation and what use cases or improvements did you notice?
Any insights or relevant experiences would be appreciated, whether it’s related to improving accuracy, response time, or optimizing for specific user scenarios. Also, are there any tools or libraries you found particularly helpful for managing and analyzing this kind of data?
Looking forward to hearing your thoughts and thanks in advance!