Before posting I did read the above post.
My goal is to present about two pages of data with about a 2 paragraph summery. I have a collection of these and I want to train completion to read input data and to summarize it in my example format. Do I need to make a fine-tuning set then create a divinci instance trained on the data then run a completion on against that specifically trained divinci? Am I going down the right path?
It seems that the dataset I am using is pretty large, so from the docs above if I understand correctly, I add the raw data as the prompt and my summarized version of the data as the completion correct?
Anyone know the limits on these prompts?
I expecting the finetuning can also have Divinci respond in a peticular way / format Hopefully
Thank you for any help!
Update I was able to make a little progress in getting my finetuning AI generated. I am reviewing the DOC and my api call dose not play nice targeting my newly trained AI in the model param of completion. I think I may have uploaded my training file (About 400kb) a few times because I got charged 9 dollars with divinchi, I may have done it only two to three times so Ill have to watch how often I click that button
Update2 I see two now and each one cost 330,104 trained tokens ($9.90)
Is there a way to calculate tokens before executing it? Just to ballpark cost?
So I created about 13 samples and fine tuned a curie. I know 13 is a small amount but I replicated one of the train files specifically in the playground with the newly trained curie and the format wasnt correct. If I need 100s if not 1000’s of examples then Ill have to code some web spider to format this data out as it will be pretty rough doing it by hand =\
Thanks, the last couple of days I’ve been on youtube falling asleep to GPT3 fine tuning searching and I did not see this at all on the results I got. Ill give this ago! The videos on youtube are helpful but they seem to mostly stick to linux python, I am writing my own .NET library to work with openAI (for the exp) so getting info at that level is pretty rough when it comes to video sources so Ill take any help I can get.
Currently, I am spidering the dataset Im going to use to train curie and am pushing around 700 examples at the moment. Im trying to find a way to generate more, any who Ill review this content and come back with other questions I have.