New updates from Logan, fine tune your finetunes is here!
7 Likes
If you didnât see it, thereâs a new fine-tune graphic interface next to the Playground, too.
5 Likes
What does Training Loss mean? The training loss at the beginning of training was about 3 points. After 100 steps of training, it reached about 0.2. Why is this? I have a total of 1,500 pieces of data, and the Epochs are 4.
Training loss is how well the model is fitting your training data. Smaller the number, the more fitted the model is to your data.
But you donât want it to be perfectly fitted, because this could lead to a model that has no ability to generalize. You would see this as a high validation loss, which is how well the model works on data it has never seen or been trained with.
4 Likes
âWhere more or fresher data becomes availableâ - I think that this approach of encouraging fine-tuning for the application of knowledge retrieval really needs to be rolled back. Thereâs been a big influx of âhow I trainâ and âwhy doesnât he workâ because of new interest being put out there.
Documentation needed to prevent misapplication:
- Fine tune success stories using gpt-3.5-turbo for specific applications.
- Cases where the example prompt itself doesnât already fulfill the output.
- Cases where only an identity shapes the new unprompted behavior and logic.
- Example conversation samples, numbers of samples and variations, epochs.
- Demonstration of realized performance on closed-domain and out of domain inputs.
- Techniques for management of existent ChatGPT-style chat weights vs your desired application, integrating with or overcoming chat fine-tuning.
- Failures or misuse which cannot be overcome by the best techniques.
- Reminding that use will cost 8x, a price that canât be reduced for content in/data out.
Better than: âexperiment with this at $400/50 million tokens/epochsâ.
You are correct. The objective of my training is to provide accurate information, but it is not possible for me to cover every possible question. Hence, the more aligned the query is with the available data, the better equipped I am to answer similar questions.
1 Like
Before anyone goes down the fine-tune route, first read this from the docs:
So a good use of a fine-tune is for classification, since I canât fit all the required examples and variations in the prompt. The lower latency is a plus!
But for accurate information, that is usually handled by Retrieval Augmented Generation, or âRAGâ, using embeddings typically, to retrieve your accurate information for the LLM to use in itâs response back.
To get this on a fine-tune, you tend to see disappointed folks, because the model is usually undertrained for this task with a fine-tune.
You can, however, overtrain the fine-tune (higher epochs of 16 or more) and it will repeat it back. But beware, this comes at a big issue of the model not being able to generalize outside of the training data. So it only works if you can ensure the inputs are highly correlated with the training data.
3 Likes
I agree with Curt, it seems like what you want may be RAG instead of fine tuning:
You could try using the RAG approach first, that should get you much better accuracy on the information, then use fine tuning if necessary to adjust the phrasing. This also has the added bonus that you can update the data without updating the fine tune.
2 Likes
So far, I use Gpt4+Langchain+prompt. Thank you, you are right.
N2U via OpenAI Developer Forum <notifications@openai1.discoursemail.com>äş2023ĺš´9ć22ćĽ ĺ¨äş02:36ĺéďź
1 Like
I see the new Fine-Tune tab. However, I do not see any updates there or in the documentation on how to fine-tune my already fine-tuned model.
For example, I created a fine-tune model - ft:gpt-3.5-turbo-0613:personal::abcdef. How can fine-tune this model again? Should I pass âft:gpt-3.5-turbo-0613:personal::abcdefâ instead of âgpt-3.5-turboâ when creating a new fine-tune model job?
1 Like
Documentation has been updated:
Can I continue fine-tuning a model that has already been fine-tuned?
Yes, you can pass the name of a fine-tuned model into the model parameter when creating a fine-tuning job. This will start a new fine-tuning job using the fine-tuned model as the starting point.
So instead of gpt-3.5-turbo, your current fine-tune model name. Then a new model is also made.
3 Likes
Thank you for your response. I tried the API by changing the model parameter to my fine-tuned model and it worked. I am hoping the documentation will get updated soon. Great work overall by the team!
{
"training_file": "xyz",
"model": "ft:gpt-3.5-turbo-0613:personal::abcdef",
"suffix": "ft-2"
}
2 Likes
When continuing a model that has already been fine-tuned, do we only get charged for the new data input during the fine-tuning job? (Example: I have a model fine-tuned with 50 sets and I need to add 30 more and continue fine-tuning the same model)
Thatâs a good question, and knowing how it operates, you can certainly strategize.
Training is performed, and you are charged by, the tokens in a training file, multiplied by the number of epochs that are performed, which are repeated passes of your training data through the reinforcement learning algorithm.
The auto mechanism might, for example, select 8 epochs for the size of your training file, where you can then only at the end see the effect of learning and validation.
You can instead specify an initial ân_epochsâ hyperparameter yourself, a lower number like 4, to get a feel for how the learning is progressing by viewing the batch report (and the effect of other learning rate hyperparameters) and then continue from that base making other models at small continuation increments such as 2 epochs, each time deepening the effect and paying only for the token passes used.
You can also set a prefix so the name of the model includes some more identifying info, like âebot-p2â for the second pass.
This will let you find which âversionâ is most effective, without overfitting ( where the AI model can only do what you trained on and even only follows the specific examples without inferring similar cases.)
1 Like
Hi, I notice that even when I provided my existing model when do fine-tuning, but when finish, it return different model name in job.fine_tuned_model, is this expected behavior?