I’m doing tests teaching the AI about a specific technology, to do that I’m getting data from the technology documentation and from blog tutorials explaining that tech, I’m placing those data into completions in a data set for fine-tuning.
Sometimes I get a large amount of data with a some paragraphs explaining a specific topic and I place it in a completion for a prompt, but I’m just wondering what is the most efficient way to do that, should I place in one single completion large information explaining a specific topic or should I split that information into multiple completions? Is there any difference on those?
Hey, welcome to the forum.
Once you send a prompt and get a completion, you cannot send another one that will “remember” the first prompt and completion.
If you want to “feed the AI” large amounts of data, you want to look into fine-tuning. You can search the forum here for lots of great posts or check on the DOCS on the OpenAI site.
Hope this is helpful.
1 Like
I’m already doing that using fine-tuning, I just forgot to mention in the question, I edited it to add that.
My question is what would be the better way to do that, if it’s by adding large completions about the same topic or splitting it into multiple completions/prompts.
It depends what kind of output you’re wanting from the fine-tuned model.
If you want longer results, I would put as much as you can in the completion of the dataset. On the other hand, if you want shorter results, give examples of that in your dataset.
1 Like