Sometimes the completion I get from the fine-tuned model differs from the sheet that I have converted to jsonl then uploaded through CLI.
Instead of yes, I get no, and vice versa. The answers should be exactly the same.
How to make the answers exact (I tried setting up the temperature to zero, but no vail.
did you provide answers to the questions and uploaded. and did you ask the same question
What model did you fine-tune from? How many samples? What are your API settings (temperature, etc…)
Yes. The prompt and completion is as follows {“prompt”:“Is ‘tea wholesaler’ relevant to teadepot.com?“,“completion”:” Yes\n”}. And by asking the same question sometimes I receive no as an answer.
I have used Davinci. and the data is about 400 samples. And I have tried to use different temperature settings no no vail
Are you using the \n for a stop-word?
You might try to up it to 800 samples and go again.
Good luck!
Yea I used \n. I uploaded huge data
Now the answers are coming right for 90% of the time which is good. (by giving the model huge data). Thanks for the help!
Regards
Prakeet Singh