Hello i started experimenting with fine tuning today through the web interface that openai has created. I provided 2 files one for training and one for validation. The most important issue is: I do not get a validation loss value, only training loss, why?
my second question: why for the love of god, when you are at the create step, the final one, you just get a “not enough funds” and they do not have a cost? they have access to the files and can tokenize them and provide a rough cost. Why do I have to “imagine” how much this is gonna cost?
As to your first point: Are you sure that you have properly uploaded the validation file prior to initiating the fine-tuning job?
If successfully uploaded, you should normally see all three values, i.e. the training loss, the validation loss, and the full validation loss in the UI.
Other than some unintentional mistake during upload and/or clicking accidentally on the None button in the section Validation data during the process, no immediate other idea springs to mind what might have caused this.
Thank you for your reply. Yes, I think i did everything alright. I did one more training job, as you can see in the new screenshot, just altered the learning rate. Now, i also get a value for full validation loss, but still not three values.
What if you download the fine tuning results file from the files endpoint (likely needing base64 decoding to CSV plain text) - do you have validation results there? You can create your own visualization from that.
Yeah, did that just now, but there is nothing related to validation loss in there:
step,train_loss,train_accuracy,valid_loss,valid_mean_token_accuracy
1,2.2987,0.76243,
2,2.13593,0.78333,
3,2.33327,0.74586,
4,2.31516,0.77273,
5,2.34954,0.74011,
6,2.20884,0.75275,
7,2.31571,0.76836,
8,2.08845,0.77901,
9,2.20509,0.76111,
If it is not urgent, you might want to get in touch with OpenAI support and see if they are able to identify the root cause of the issue. It seems unusual.
I have same problem. I uploaded the training data and validation data from Python properly. It showed up in my dashboard. And then I set the training data and the validation data for training. I still don’t have a clue why this happened.
