Question about fine tuning

Hey, I was wondering if it would be possible to fine tune GPT3 on a bunch of content about Skyrim for example, then use it to answer questions about the game/explain lore, backsotry and game mechanics to you?

I know when you fine tune you need to break the data into prompts and responses, but could you use another GPT model to convert the online content you find into a question and answer (prompt and response) format?
I am new to fine tuning but I’d love be able to set GPT up to be a professional on one particular topic. Any advice or research references would be wounderful! :grin:

Hardest problem will be getting the raw data

I don’t fully understand what you mean, do you mean getting the text itself or finding text that is in a useful form for training GPT?

I’m a fan of your videos by the way, keep it up :grin:

1 Like

The text. I’m imagining that most Skyrim data is in images and videos.

Also, thank you, glad you like them

Hi @joshacc22 ,

Welcome to the OpenAI community.

GPT-3 was trained on data till some month in 2019. So if the game/franchise/series you mention was released prior to that and had some wiki available, chances are GPT-3 already knows about it.

This is my hypothesis, to test it, prompt davinci engine with something like:

Human: You are an expert on <Name of the topic>. Can you please answer some of my questions?
AI: Sure. I’ll do my best.
Human:<Your question>

I’m curious if this works. Let us know of the results.


Latest instruct contains data up till 2021 at least

1 Like

How about the Answers endpoint? You can upload Documents (so it knows you’re talking about Skyrim’s fireballs, rather than D&D fireballs).

Aboud documents for fine tuning, the format is different from the prompt/response example in other fine tunings. You only have to give items of text, so it does not need to be in the form of a prompt/response, if you are thinking you will need a lot of extraction and transformation. I think you could literally copy and paste straight paragraphs (if you have the permissions/rights to, of course)

1 Like

Wow okay thanks! I haven’t played with the answers endpoint before. I will check it out, thankyou! :grin: