Fine Tuning GPT-3 for Consistent Output Format

[cyrilg]

I am trying to use the Open AI API to create quiz questions with three incorrect answers and one correct answer. The prompt I use is:

Write a quiz on ${quiz_topic} with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer. Write each answer on a separate line

I would like to parse GPT-3’s answer but the format is inconsistent. For instance, if quiz_topic is “literature”, I get the following response:

Q1. Who wrote the novel "The Catcher in the Rye"?

A. J.D. Salinger
B. Ernest Hemingway
C. F. Scott Fitzgerald
D. Mark Twain

This is the correct format. But if I use “U.S History”, I get the following output:

Q1: Who was the first president of the United States?

A1: George Washington Jr., Thomas Jefferson, John Adams, George Washington

This is a different format. So my goal is to fine-tune it using the article here

I have created 200 line of a training set. Here is a sample:

{"prompt":"Write a quiz on general knowledge with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer.Write each answer on a separate line \n\n---\n\n","completion":" Q. What is the Italian word for "tomato"?\nA. Aglio\nA. Cipolla\nA. Peperoncino\nA. Pomodoro\n\nQ. Which restaurant's mascot is a clown?\nA. Whataburger\nA. Burger King\nA. Sonic\nA. McDonald's\n\nQ. Which of these colours is NOT featured in the logo for Google?\nA. Yellow\nA. Blue\nA. Green\nA. Pink\n\nQ. In 2013 how much money was lost by Nigerian scams?\nA. $95 Million\nA. $956 Million\nA. $2.7 Billion\nA. $12.7 Billion\n\nQ. What is the name of Poland in Polish?\nA. Pupcia\nA. Polszka\nA. Póland\nA. Polska\n\n \n\n###\n\n"}
{"prompt":"Write a quiz on books with 5 questions. Each question has 3 incorrect answers and 1 correct answer. The correct answer is always the last answer.Write each answer on a separate line \n\n---\n\n","completion":" Q. What is the name of Eragon's dragon in "Eragon"?\nA. Glaedr\nA. Thorn\nA. Arya\nA. Saphira\n\nQ. In the "The Hobbit", who kills Smaug?\nA. Bilbo Baggins\nA. Gandalf the Grey\nA. Frodo\nA. Bard\n\nQ. What is Hermione Granger's middle name?\nA. Jane\nA. Emma\nA. Jo\nA. Jean\n\nQ. According to The Hitchhiker's Guide to the Galaxy book, the answer to life, the universe and everything else is...\nA. Loving everyone around you\nA. Chocolate\nA. Death\nA. 42\n\nQ. What is the name of the three headed dog in Harry Potter and the Sorcerer's Stone?\nA. Spike\nA. Poofy\nA. Spot\nA. Fluffy\n\n \n\n###\n\n"}

When I run the validation tool with the command

openai tools fine_tunes.prepare_data -f training.jsonl

I get the following message

- All prompts start with prefix `Write a quiz on `. Fine-tuning doesn't require the instruction specifying the task, or a few-shot example scenario. Most of the time you should only add the input data into the prompt, and the desired output into the completion

I don’t understand why I must remove “Write a quiz on”. So I seem to have misunderstood how to fine-tune a model for consistent formatting. Can anybody shed a light on how to fine-tune it to make sure I get the same formatting with the same prompt?

[wfhbrian]

I was impressed how asking to make the last answer the correct completely stumps ChatGPT and text-davinci-003.

If you can change the format, this prompt should work without fine-tuning:

Write a five multiple-choice questions on U.S. History. Please use the format template. Do not repeat answers.
---BEGIN FORMAT TEMPLATE---
${QUESTION}
A) ${ANSWER 1}
B) ${ANSWER 2}
C) ${ANSWER 3}
D) ${ANSWER 4}
Correct Answer: ${CHOICE A, B, C, or D}
---END FORMAT TEMPLATE---
---BEGIN QUESTIONS---

[cyrilg]

Thank you very much Brian, that worked perfectly. I wasn’t aware of chatGPT’s templating feature.
I tested it on 50 random topics, and the output is always consistent.

[raoul]

@wfhbrian I’ve seen a lot of sample prompts but nothing like this format template. Can you share if there is more information on writing templates like this and where I can find it?

I have a series of classification questions that I would liek to output in a format denoting the first character of a choice, and it’s giving me a hard time.
Example:

Evaluate the scipt below and identify each of the two options.

1. Apples or Oranges?
2. Trains or Planes?

Response: {OT}

[wfhbrian]

I’ve been checking out prompts as much as possible for the past year so I’ve picked up lots of tips along the way which lead to how I do things now.

I write a lot of my prompts in a similar way. I might write a blog post about the thinking that goes into them. If you have any ideas you’d like me to explore, let me know and I’ll see what I can do.

Regarding that specific prompt, what do you mean by OT? If it isn’t something as standard as multiple-choice questions, it would help to add a few examples.

[raymonddavey]

I’ve been doing similar things with square brackets. It seems to evaluate things within brackets as sub tasks or places where it needs to do something specific.

The dollar sign probably helps a lot, as it is a common prefix for variable names. I hadn’t considered that, and might try a few experiments.

[raoul]

Thanks Brian. Here is another (Synthetic) example.

So for a story I am trying to analyze the character types in the story, N for Narrator or C for Conversational, the Gender of the character F for Female or M for Male. Then a unique number for each character in the script. I am abbreviating the names to try to stuff more words in the reponse before hitting the token limit for text-davinci-003.

On this sentence:
John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

I would like the following output:
[NM1] John stopped to take it all in and said, [CM2]“Wow, look at all these people! It’s so exciting to celebrate our freedom.”

– Prompt:
Analyze the story below. For every sentence and quote, determine if is of a Narrative or Conversational style. Detemine the gender of the speaker Male or Female. Generate a unique number for each character in the story. Return the full script in your response and prefix every sentence and quote with your responses in the format [NM1] or [CF2].

John, Jenny, and Jack were all walking down the street on Independence Day, excited to get to the festivities. Jenny was wearing a bright yellow dress, waving her small American flag proudly.

John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

Jenny smiled and said, “I’m so happy to be an American and to celebrate this day.”

Jack chimed in, “I’m excited to see the fireworks later. I can already hear the sound of the marching band in the distance.”

John smiled and said, “I know, it’s great! We should go find a good spot to watch the parade.”

The three of them continued walking, taking in the sights and sounds of the day. Everywhere they looked, people were waving flags and wearing red, white, and blue. They stopped to watch a group of children singing “The Star-Spangled Banner,” and Jenny began to tear up.

Jack put his arm around her and said, “It’s okay, Jenny, I’m proud of you.”

John said, “We’re all proud of you. This is an important day in American history, and we’re happy to be able to celebrate it together.”

[wfhbrian]

@raoul This might help. Let me know how it works.

Analyze the following story. For every sentence and quote, determine if it is of a Narrative or Conversational style. Determine the gender of the speaker Male or Female. Generate a unique number for each character in the story. Return the full script in your response and prefix every sentence and quote with your responses in the format [NM1] or [CF2].
---BEGIN EXAMPLE---
{
	"John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”": "[NM1] John stopped to take it all in and said, [CM2]“Wow, look at all these people! It’s so exciting to celebrate our freedom.”"
}
---END EXAMPLE---
---BEGIN STORY---
John, Jenny, and Jack were all walking down the street on Independence Day, excited to get to the festivities. Jenny was wearing a bright yellow dress, waving her small American flag proudly.

John stopped to take it all in and said, “Wow, look at all these people! It’s so exciting to celebrate our freedom.”

Jenny smiled and said, “I’m so happy to be an American and to celebrate this day.”

Jack chimed in, “I’m excited to see the fireworks later. I can already hear the sound of the marching band in the distance.”

John smiled and said, “I know, it’s great! We should go find a good spot to watch the parade.”

The three of them continued walking, taking in the sights and sounds of the day. Everywhere they looked, people were waving flags and wearing red, white, and blue. They stopped to watch a group of children singing “The Star-Spangled Banner,” and Jenny began to tear up.

Jack put his arm around her and said, “It’s okay, Jenny, I’m proud of you.”

John said, “We’re all proud of you. This is an important day in American history, and we’re happy to be able to celebrate it together.”
---END STORY---
---BEGIN---

[ruby_coder]

Writing like this is not my domain, but your prompt looks good from this vantage point: @wfhbrian

[raoul]

Thank so much, will work on it and let you know!

[lemoncoral]

@wfhbrian Hi, Brian. There is another problem I’d like to get some help from you. I want to tell ChatGPT to translate some content, and respond with JSON-formatted string array. Using ChatGPT 3.5-turbo, the result is not always stable. And the format is not always what I want.

My system prompt is:

As a translator, your task is to translate giving contents into Simplified Chinese.Follow these steps:

step1: Translate every line content separated by ‘###’. If you cannot translate the text, just leave it as it is. Do not combine any multiple line into one. Just translate the line content as it is. If the translated content include double quotes, use single quotes instead.

step2: Respond with a JSON array format, with each item contains the translated line content

My test input is:

As a translator, your task is to translate giving contents into Simplified Chinese.Follow these steps:###
step1: Translate every line content separated by '#'. If you cannot translate the text, just leave it as it is. Do not combine any multiple line  into one. Just translate the line content as it is. If the translated content include double quotes, use single quotes instead.###
step2: Respond with a JSON array format,  with each item contains the translated line content

ant the output always are like:
[
“作为译者，您的任务是将给定的内容翻译成简体中文。”,
“按照以下步骤进行：”,
“第1步：翻译每行文本，用#分隔。如果您无法翻译文本，请将其保留不变。不要将多个行合并为一行。只需按原样翻译行内容即可。如果翻译内容包含双引号，请改用单引号。”,
“第2步：以JSON数组格式回复，每个项包含翻译的行内容。”
]

As we can see, the original text ‘Follow these steps:’ is in the first block, so that it should appear in the
first item in the result array. But seems ChatGPT put it into a separate item in the result array.

How can I instruct ChatGPT to respond with my rule or format?

Thanks very much!