Teaching GPT3 about current trends - Finetuning

API
1

GPT trained data is probably out of date and would not understand the current trends or new studies in 2021, What would be the best way to finetune a model with current information and be able to ask the model questions on this new trained data ?

1 Like
2

I think the “empty prompt” method detailed in the documentation is the way to go.

2 Likes
3

Welcome @munye :blush:

I agree with @daveshapautomator, you can fine-tune a model to have knowledge of current events using open ended generation, as described here: OpenAI API

Also take into account the vast volume GPT-3 was trained on, training on “everything” that’s happened over the last 2 years, to the same level GPT-3 was originally trained on, will not fit into a 200Mb fine-tuning training file, but I’m sure you have a specific topic in mind, making the task more manageable.

I’ve also had good results training facts with question answer pairs (also see OpenAI API), the difference in your training file would be something along the lines of:

  • Open ended generation json line example
{"prompt": "", "completion": " COVID-19 is a pandemic that had a dramatic effect on world economy."}
  • Question-Answer json line example
{"prompt": "What is Covid-19? \nAnswer:", "completion": " It is a pandemic that had a dramatic effect on world economy.\n###\n"}
5 Likes
4

Thanks Carla, this is really helpful.

3 Likes