Hi,
I am fine tuning a model to generate email replies tailored to specific teams/users.
Fine tuning a seperate GPT-4o model for each user based on e-mail and reply pairs.
Then I am updating the fine-tuned models every month. For this I select the 100 top emails where the generated response differs the most from the actual sent out response (where the user had to edit the most). So each month there is a new training set, with a new updated model off from the previus one (this will be important later).
First of all, is this a correct approach for capturing the mistakes of the AI model?
Second I am thinking about cases when I’ll need to update the prompt of the entire model. For example I add some user preference , or GPT5 comes out next year, and I want to retrain using the new prompt or new underlying model.
So for this retraining how shall I go about?
A: Retraing new model in one step using all the email-response pairs available
B: Replaying the same training steps of initial model, and incremental updates. (This assumes I save the training files for each update)
I am leaning towards A, as a new model may behave very differently and would not even make the same mistakes. Plus B option involves a lot more complex logic and infra for storing the training steps.