Creating muliple choice questions?

API
1

Hello - i want to create 2000 multiple choice questions (with 4 possible answers, 3 incorrect and one correct)
Regarding a specific topic for watches

I want to output this questions in 6 columns to us it in an excel-sheet (question, 4 answers, correct answer)

I tried it with this logic:

      resp = openai.ChatCompletion.create(
        model = "gpt-3.5-turbo",
        # model = "gpt-3.5-turbo-16k",
        messages=[
          {"role": "system", "content": "Create 10 multiple choice questions as possible (overall 4 answers with 3 incorrect and1 correct answers, show the correct answer)"},
          {"role": "system", "content": "For the following topics: Watch manufacturing, Watch parts, Watch brands, Watch history, Watch prices"},
          {"role": "system", "content": "every answer should have 6 columns: question, answer 1 to answer 4, correct answer"},
          {"role": "assistant", "content": f"Don´t use this questions: {existQuestions}"},
          {"role": "user", "content": f"Create questions"}
          ],
        request_timeout = 600, 
        temperature = 0,
        # max_tokens = 1000
      )

But its not working correctly.
I get duplicate responses and also the output is not allways in 6 columns.

How can i do this?

2

Try doing this with chat gpt, especially if you have gpt-4 access. While it might not have exactly like the 6 columns that you want, it should be able to generate good quality examples for you.

And when it runs out, just using continue will allow it to use the above generated questions as reference, which might reduce duplication

3

What do you mean with “runs out”?
And “just using continue”?
Is this a parameter in the ChatAPI? But i can´t find any continue parameter in the documentation:

KR, Max

4
  1. Don’t provide multiple system messages
  2. Don’t capitalize words where capitalization is not appropriate
  3. Ask for the desired production in a user’s instruction
  4. Expect the same output for the same input, especially at temperature 0
  5. To avoid repetition, ask for 100 questions (or whatever runs the model output up to max), this will naturally get you originals;
  6. You provided some topics that don’t correlate to the number of outputs
  7. Give a different subject on successive runs

You should show an example output beyond its description.

Here is a system prompt for API, no user role needed:

// AI role
Your role and purpose is now as an AI multiple-choice automated quiz writer, a backend text processor. AI response must be only a valid CSV output file meant for data import. Required: no AI commentary or tutorial, no AI introduction nor markdown. AI generated output will be directly written to a csv file.

// quiz specifications
- quiz purpose: evaluate skills of a jewelry apprentice
- quiz domain: watch manufacturing, timepiece parts, wristwatch brands, history, prices
- quiz format: 4 unnumbered answers, with one correct and three deceptively incorrect, random order
- quiz output: single row csv per question, column key: question number, question, answer 1, answer 2, answer 3, answer 4, correct answer
- quiz length: 100 questions

// format of csv output:
1, The most prestigious Swiss watch maker?, Omega, Rolex, Tissot, Seiko, Rolex
2, …

Output:

1,What is the most prestigious Swiss watch maker?,Omega,Rolex,Tissot,Seiko,Rolex
2,Which brand is known for its iconic Royal Oak model?,Patek Philippe,Rolex,Audemars Piguet,Breitling,Audemars Piguet
3,What is the primary function of a watch bezel?,To protect the dial,To adjust the time,To display the date,To measure the depth of water,To protect the dial
4,Which material is commonly used for watch cases?,Stainless steel,Plastic,Aluminum,Wood,Stainless steel
5,What is the purpose of a watch crown?,To wind the watch,To adjust the time,To display the date,To measure the depth of water,To wind the watch
6,Which brand is known for its Submariner model?,Rolex,Omega,Tissot,Seiko,Rolex
7,What is the function of a watch movement?,To power the watch,To protect the dial,To adjust the time,To measure the depth of water,To power the watch
8,Which brand is known for its Speedmaster model?,Omega,Rolex,Tissot,Seiko,Omega
9

The numbers I added so the AI doesn’t stop until it reaches the goal.

(also, ChatGPT 3.5 is unstoppable chatty garbage, only describing how I could do such and putting just a few examples in a code block. I shouldn’t need to also demonstrate a jailbreak to achieve a particular output.)

5

Thats amazing - thx a lot for your insights!

6

I tried it with the model: gpt-4
And i allways create 50 QAs per request.
But i have the problem that i get way to many equal answers at output, which i allready have from previous requests.

Any way to handle this so i can achieve the several thousands of individual QAs i would need?

7

What are “equal answers?” “identical questions?” I see “from previous requests” to inform what you might be discussing.

It is natural that if you haven’t changed the input to diversify or sub-categorize the questions generated, they will be similar.

Ask the AI for a joke, your going to get “AI’s greatest hits” from the top 5 it knows, every time.

These questions will take human intervention regardless. The AI isn’t a truth machine. Thousands of questions will likely have hundreds of poor answers. You can embed them all and then have a similarity search for each. Then some UI to review the questions and mark the worst or the duplicates produced by search.

For an inspirational AI, go here, put in some number of a bible verse, and it will search for similar so you can find the similar concepts (and plagiarism) within the book. https://christopherminson.com/demos/bible/qverse.html. You can imagine similar, but for your questions.