Hey, I was wondering if it would be possible to fine tune GPT3 on a bunch of content about Skyrim for example, then use it to answer questions about the game/explain lore, backsotry and game mechanics to you?
I know when you fine tune you need to break the data into prompts and responses, but could you use another GPT model to convert the online content you find into a question and answer (prompt and response) format?
I am new to fine tuning but I’d love be able to set GPT up to be a professional on one particular topic. Any advice or research references would be wounderful!
GPT-3 was trained on data till some month in 2019. So if the game/franchise/series you mention was released prior to that and had some wiki available, chances are GPT-3 already knows about it.
This is my hypothesis, to test it, prompt davinci engine with something like:
Human: You are an expert on <Name of the topic>. Can you please answer some of my questions?
AI: Sure. I’ll do my best.
Human:<Your question>
I’m curious if this works. Let us know of the results.
How about the Answers endpoint? You can upload Documents (so it knows you’re talking about Skyrim’s fireballs, rather than D&D fireballs).
Aboud documents for fine tuning, the format is different from the prompt/response example in other fine tunings. You only have to give items of text, so it does not need to be in the form of a prompt/response, if you are thinking you will need a lot of extraction and transformation. I think you could literally copy and paste straight paragraphs (if you have the permissions/rights to, of course)
