Fine tuning: What to do with completions that are the answer to multiple prompts?

Prompting
1

Another possibly noob question that came to me while putting together a dataset for fine-tuning (I want open-ended generation):

What happens if you have multiple prompts that have the same completion? For example the prompts “What country is Toronto in?”, and “What country has a flag with a maple leaf on it?” both have “Canada” as their completion. If I want my system to answer both of these prompts, should I split them into different prompts? e.g.

A)
{“prompt”: “What country is Toronto in?”, “completion”: “Canada”}
{“prompt”: “What country has a flag with a maple leaf on it?”, “completion”: “Canada”}

Or should I roll them into one large prompt? e.g.

B)
{“prompt”: “What country is Toronto in? What country has a flag with a maple leaf on it?”, “completion”: “Canada”}

Thanks for helping me out every time :pray:

2

My hunch is different prompts would work better, but I’m just making an informed guess.

1 Like
3

Thanks for the reply. Your informed guess is better than my uninformed guess!

4

I would like to fine-tune with multi-question prompts. Did this eventually work for you?