I’m currently using instruct beta v3 which is remarkably fast. But the details that makes me surprised is about this model knowing anything about Covid-19 which I think the OpenAI database has been updated up to 3rd quarter of 2020 (probably). What do you think?
@muhammadlufti You are correct, gpt3 base models have only been pretrained up to pre-covid internet. If you would like to update your own models, completion-only finetuning on 2020+ data may help, or combining gpt3 with an external knowledge base (/answers endpoint).
Right, hopefully OpenAI will managed to update and pretrained the model the databases up to date since finetuning has lot more to explore.
@m-a.schenk true, though it’s only finetuned on github code. Seems to have picked up on covid through github comments though.