Fine Tuning GPT-3 for Consistent Output Format

cyrilg · February 3, 2023, 8:19am

I am trying to use the Open AI API to create quiz questions with three incorrect answers and one correct answer. The prompt I use is:

Write a quiz on ${quiz_topic} with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer. Write each answer on a separate line

I would like to parse GPT-3’s answer but the format is inconsistent. For instance, if quiz_topic is “literature”, I get the following response:

Q1. Who wrote the novel "The Catcher in the Rye"?

A. J.D. Salinger
B. Ernest Hemingway
C. F. Scott Fitzgerald
D. Mark Twain

This is the correct format. But if I use “U.S History”, I get the following output:

Q1: Who was the first president of the United States?

A1: George Washington Jr., Thomas Jefferson, John Adams, George Washington

This is a different format. So my goal is to fine-tune it using the article here

I have created 200 line of a training set. Here is a sample:

{"prompt":"Write a quiz on general knowledge with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer.Write each answer on a separate line \n\n---\n\n","completion":" Q. What is the Italian word for &quot;tomato&quot;?\nA. Aglio\nA. Cipolla\nA. Peperoncino\nA. Pomodoro\n\nQ. Which restaurant&#039;s mascot is a clown?\nA. Whataburger\nA. Burger King\nA. Sonic\nA. McDonald&#039;s\n\nQ. Which of these colours is NOT featured in the logo for Google?\nA. Yellow\nA. Blue\nA. Green\nA. Pink\n\nQ. In 2013 how much money was lost by Nigerian scams?\nA. $95 Million\nA. $956 Million\nA. $2.7 Billion\nA. $12.7 Billion\n\nQ. What is the name of Poland in Polish?\nA. Pupcia\nA. Polszka\nA. P&oacute;land\nA. Polska\n\n \n\n###\n\n"}
{"prompt":"Write a quiz on books with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer.Write each answer on a separate line \n\n---\n\n","completion":" Q. What is the name of Eragon&#039;s dragon in &quot;Eragon&quot;?\nA. Glaedr\nA. Thorn\nA. Arya\nA. Saphira\n\nQ. In the &quot;The Hobbit&quot;, who kills Smaug?\nA. Bilbo Baggins\nA. Gandalf the Grey\nA. Frodo\nA. Bard\n\nQ. What is Hermione Granger&#039;s middle name?\nA. Jane\nA. Emma\nA. Jo\nA. Jean\n\nQ. According to The Hitchhiker&#039;s Guide to the Galaxy book, the answer to life, the universe and everything else is...\nA. Loving everyone around you\nA. Chocolate\nA. Death\nA. 42\n\nQ. What is the name of the three headed dog in Harry Potter and the Sorcerer&#039;s Stone?\nA. Spike\nA. Poofy\nA. Spot\nA. Fluffy\n\n \n\n###\n\n"}

When I run the validation tool with the command

openai tools fine_tunes.prepare_data -f training.jsonl

I get the following message

- All prompts start with prefix `Write a quiz on `. Fine-tuning doesn't require the instruction specifying the task, or a few-shot example scenario. Most of the time you should only add the input data into the prompt, and the desired output into the completion

I don’t understand why I must remove “Write a quiz on”. So I seem to have misunderstood how to fine-tune a model for consistent formatting. Can anybody shed a light on how to fine-tune it to make sure I get the same formatting with the same prompt?

wfhbrian · February 3, 2023, 5:37pm

I was impressed how asking to make the last answer the correct completely stumps ChatGPT and text-davinci-003.

If you can change the format, this prompt should work without fine-tuning:

Write a five multiple-choice questions on U.S. History. Please use the format template. Do not repeat answers.
---BEGIN FORMAT TEMPLATE---
${QUESTION}
A) ${ANSWER 1}
B) ${ANSWER 2}
C) ${ANSWER 3}
D) ${ANSWER 4}
Correct Answer: ${CHOICE A, B, C, or D}
---END FORMAT TEMPLATE---
---BEGIN QUESTIONS---

cyrilg · February 3, 2023, 8:24pm

Thank you very much Brian, that worked perfectly. I wasn’t aware of chatGPT’s templating feature.
I tested it on 50 random topics, and the output is always consistent.

raoul · February 12, 2023, 6:05pm

@wfhbrian I’ve seen a lot of sample prompts but nothing like this format template. Can you share if there is more information on writing templates like this and where I can find it?

I have a series of classification questions that I would liek to output in a format denoting the first character of a choice, and it’s giving me a hard time.
Example:

Evaluate the scipt below and identify each of the two options.

Apples or Oranges?
Trains or Planes?

Response: {OT}

wfhbrian · February 12, 2023, 7:02pm

I’ve been checking out prompts as much as possible for the past year so I’ve picked up lots of tips along the way which lead to how I do things now.

I write a lot of my prompts in a similar way. I might write a blog post about the thinking that goes into them. If you have any ideas you’d like me to explore, let me know and I’ll see what I can do.

Regarding that specific prompt, what do you mean by OT? If it isn’t something as standard as multiple-choice questions, it would help to add a few examples.

raymonddavey · February 12, 2023, 7:07pm

I’ve been doing similar things with square brackets. It seems to evaluate things within brackets as sub tasks or places where it needs to do something specific.

The dollar sign probably helps a lot, as it is a common prefix for variable names. I hadn’t considered that, and might try a few experiments.

raoul · February 12, 2023, 8:36pm

Thanks Brian. Here is another (Synthetic) example.

So for a story I am trying to analyze the character types in the story, N for Narrator or C for Conversational, the Gender of the character F for Female or M for Male. Then a unique number for each character in the script. I am abbreviating the names to try to stuff more words in the reponse before hitting the token limit for text-davinci-003.

On this sentence:
John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

I would like the following output:
[NM1] John stopped to take it all in and said, [CM2]“Wow, look at all these people! It’s so exciting to celebrate our freedom.”

– Prompt:
Analyze the story below. For every sentence and quote, determine if is of a Narrative or Conversational style. Detemine the gender of the speaker Male or Female. Generate a unique number for each character in the story. Return the full script in your response and prefix every sentence and quote with your responses in the format [NM1] or [CF2].

John, Jenny, and Jack were all walking down the street on Independence Day, excited to get to the festivities. Jenny was wearing a bright yellow dress, waving her small American flag proudly.

John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

Jenny smiled and said, “I’m so happy to be an American and to celebrate this day.”

Jack chimed in, “I’m excited to see the fireworks later. I can already hear the sound of the marching band in the distance.”

John smiled and said, “I know, it’s great! We should go find a good spot to watch the parade.”

The three of them continued walking, taking in the sights and sounds of the day. Everywhere they looked, people were waving flags and wearing red, white, and blue. They stopped to watch a group of children singing “The Star-Spangled Banner,” and Jenny began to tear up.

Jack put his arm around her and said, “It’s okay, Jenny, I’m proud of you.”

John said, “We’re all proud of you. This is an important day in American history, and we’re happy to be able to celebrate it together.”

wfhbrian · February 14, 2023, 12:24am

@raoul This might help. Let me know how it works.

Analyze the following story. For every sentence and quote, determine if it is of a Narrative or Conversational style. Determine the gender of the speaker Male or Female. Generate a unique number for each character in the story. Return the full script in your response and prefix every sentence and quote with your responses in the format [NM1] or [CF2].
---BEGIN EXAMPLE---
{
	"John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”": "[NM1] John stopped to take it all in and said, [CM2]“Wow, look at all these people! It’s so exciting to celebrate our freedom.”"
}
---END EXAMPLE---
---BEGIN STORY---
John, Jenny, and Jack were all walking down the street on Independence Day, excited to get to the festivities. Jenny was wearing a bright yellow dress, waving her small American flag proudly.

John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

Jenny smiled and said, “I’m so happy to be an American and to celebrate this day.”

Jack chimed in, “I’m excited to see the fireworks later. I can already hear the sound of the marching band in the distance.”

John smiled and said, “I know, it’s great! We should go find a good spot to watch the parade.”

The three of them continued walking, taking in the sights and sounds of the day. Everywhere they looked, people were waving flags and wearing red, white, and blue. They stopped to watch a group of children singing “The Star-Spangled Banner,” and Jenny began to tear up.

Jack put his arm around her and said, “It’s okay, Jenny, I’m proud of you.”

John said, “We’re all proud of you. This is an important day in American history, and we’re happy to be able to celebrate it together.”
---END STORY---
---BEGIN---

ruby_coder · February 14, 2023, 3:52am

Writing like this is not my domain, but your prompt looks good from this vantage point: @wfhbrian

raoul · February 14, 2023, 9:10pm

Thank so much, will work on it and let you know!