I’m relatively new to fine-tuning models and working on a project that involves generating specific outputs based on structured input data. I’ve been experimenting with fine-tuning a GPT-4-based model, and I’m attempting to handle two different tasks using equal sets of training data.
So far, I’ve used around 60 data sets, split equally between the two tasks. While the model performs well on one task, it consistently struggles with the second. It often misses key phrases and outputs that were frequently present in the training data.
I’ve tried various tweaks but can’t seem to get the model to perform consistently across both tasks. Any advice on best practices or potential strategies for balancing performance between multiple tasks during fine-tuning would be greatly appreciated!
What’s the specific objective of your fine-tuning project? Are you looking to get the model to respond in a specific format / style or are you trying to get the model to return content from your training data?
In case the latter applies, please be aware that this could not be achieved with fine-tuning and you would have to resort to other measures, such as RAG.
In case of the former, then it would be helpful if you could share a few more details about the two tasks, the instructions you are providing etc… As a general point, if the two tasks are very different in nature, then it is not recommended to combine them under one fine-tuned model and instead rather create two separate models.
What is the logic for the generation of the billing codes? Is there a specific pattern that the model can pick up on based on the training data?
Please bear in mind that model will not systematically remember information included in your training data. So if there are specific information that the model needs in order to generate the billing codes, then you still need to include that information at inference in the context as part of the system or user message.
For more specific guidance, it would help if you could share example instructions for both tasks including the system message you are using (if applicable)?
I can’t share the data here because of few reasons. But I can tell that the I’ve been including the information in the user message. What other options are there rather than RAG ?
sorry