API response is not stable

I am using chatgpt completions API .
Parameter i am sending are here
array(“model” => “gpt-3.5-turbo-instruct” ,
“prompt” => “5 mutlichoice question on science with answer in json format response.” ,
//“temprature” => 0.7,
“max_tokens” => 1400 ,
“top_p” => 1,
“frequency_penalty” => 0,
“presence_penalty” => 2.0
);
i am asking for proper json format but some times its gives text and different format. So please help me and do let me know solution.

Thanks

Hello! Welcome to the forum.

You need a long system prompt.

You might want to “one-shot” it (which means, basically, give it one example of a request and what you want in reply…

It’s more tokens, but more stable usually…

3 Likes

Consider adding a more detailed input prompt to achieve consistent output.

Example Prompt:

Generate 5 multiple-choice questions on science, each with 4 options labeled A, B, C, D, and indicate the correct answer. Format the response in JSON, as shown in the example below:

Example:
[
  {
    "question": "What is the boiling point of water?",
    "options": {
      "A": "100°C",
      "B": "90°C",
      "C": "80°C",
      "D": "110°C"
    },
    "correct_answer": "A"
  }
]

Follow this format for the 5 science multiple-choice questions.

Also Adjust Parameters : While your parameters seem generally fine, you might not need a presence_penalty of 2.0, especially since you’re looking for structured output rather than creative content. A presence_penalty might make the model avoid repeating terms that could be necessary for your format. Consider reducing it or setting it to 0.

https://platform.openai.com/docs/api-reference/chat/create

2 Likes

Hi @Innovatix

As you told i tried but its not working.
Any other option do you have ?

Okay, may I know what issue you are facing?

Could you share your code and the output to better assist you?

1 Like

Code is here

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://api.openai.com/v1/completions');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
$postdata = array("model" => "gpt-3.5-turbo-instruct", 
"prompt" => "5 mutlichoice question on science with answer like question, options and correct_answer json format" ,
//"temprature" => 0.7,
"max_tokens" => 1400 ,
"top_p" => 1,
 "frequency_penalty" => 0,
 //"presence_penalty" => 2.0
);
$postdata = json_encode($postdata);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);

$headers = array();
$headers[] = 'Content-Type: application/json';
$headers[] = 'Authorization: Bearer ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■pL3jL5tA';
//$headers[] = 'Openai-Beta: completions=v1';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);
if (curl_errno($ch)) {
    echo 'Error:' . curl_error($ch);
}
//echo '<pre>'; print_r(json_decode($result));
$arr = json_decode($result);
  echo '<pre>'; print_r($arr->choices[0]->text); 

Response

{
“question”: “What is the chemical symbol for gold?”,
“options”: [“Au”, “Ag”, “Fe”, “Hg”],
“correct_answer”: “Au”
},
{
“question”: “Which planet is known as the Red Planet?”,
“options”: [“Earth”, “Mars”, “Neptune”, “Jupiter”],
“correct_answer”: “Mars”
},
{
“question”: “What is the largest organ in the human body?”,
“options”: [“Heart”, “Liver”, “Skin”, “Brain”],
“correct_answer”: “Skin”
},
{
“question”: “What is the process by which plants make their own food?”,
“options”: [“Photosynthesis”, “Respiration”, “Fertilization”, “Germination”],
“correct_answer”: “Photosynthesis”
},
{
“question”: “What type of energy is stored in plants and animals?”,
“options”: [“Nuclear energy”, “Chemical energy”, “Solar energy”, “Kinetic energy”],
“correct_answer”: “Chemical energy”
}

AFTER refreshing of 3 to 5 times its start giving text format.
json format is also wrong.

Suggestion:


Code with Your Prompt :

your prompt response

[
  {
    "question": "Which of the following is NOT a type of galaxy?",
    "options": ["Spiral galaxy", "Elliptical galaxy", "Irregular galaxy", "Dim galaxy"],
    "correct_answer": "Dim galaxy"
  },
  {
    "question": "What is the study of living organisms?",
    "options": ["Geology", "Astronomy", "Ecology", "Biology"],
    "correct_answer": "Biology"
  },
  {
    "question": "What is the force that causes objects to fall towards the earth?",
    "options": ["Gravity", "Friction", "Magnetism", "Inertia"],
    "correct_answer": "Gravity"
  },
  {
    "question": "What is the densest element on earth?",
    "options": ["Gold", "Platinum", "Lead", "Osmium"],
    "correct_answer": "Osmium"
  },
  {
    "question": "What is the process by which plants use sunlight to make food?",
    "options": ["Photosynthesis", "Respiration", "Fertilization", "Germination"],
    "correct_answer": "Photosynthesis"
  }
]

Python Code for with Chat completions and better prompt

import requests
import json

url = 'https://api.openai.com/v1/chat/completions'

headers = {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YourKEY', 
}

data = {
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": """Generate 5 multiple-choice questions on science, each with 4 options labeled A, B, C, D, and indicate the correct answer. Format the response in JSON, as shown in the example below:

          Example:
          [
            {
              "question": "What is the boiling point of water?",
              "options": {
                "A": "100°C",
                "B": "90°C",
                "C": "80°C",
                "D": "110°C"
              },
              "correct_answer": "A"
            }
          ]"""
        }
    ]
}

response = requests.post(url, headers=headers, json=data)

if response.status_code == 200:
    print(json.dumps(response.json(), indent=2))
else:
    print("Error:", response.text)

Response with the new prompt

[
  {
    "question": "Which of the following components is NOT found in the nucleus of an atom?",
    "options": {
      "A": "Proton",
      "B": "Neutron",
      "C": "Electron",
      "D": "Quark"
    },
    "correct_answer": "C"
  },
  {
    "question": "What is the process by which plants make their food called?",
    "options": {
      "A": "Respiration",
      "B": "Photosynthesis",
      "C": "Fermentation",
      "D": "Transpiration"
    },
    "correct_answer": "B"
  },
  {
    "question": "Which planet is known as the 'Red Planet'?",
    "options": {
      "A": "Venus",
      "B": "Mercury",
      "C": "Mars",
      "D": "Jupiter"
    },
    "correct_answer": "C"
  },
  {
    "question": "What is the unit of electrical resistance?",
    "options": {
      "A": "Farad",
      "B": "Watt",
      "C": "Ohm",
      "D": "Volt"
    },
    "correct_answer": "C"
  },
  {
    "question": "Which of the following is NOT a type of rock?",
    "options": {
      "A": "Igneous",
      "B": "Granite",
      "C": "Metamorphic",
      "D": "Subduction"
    },
    "correct_answer": "D"
  }
]

Sending such a poorly-specified instruction to a completion model, and using default sampling parameter value of 1.0 for both temperature and top_p, you certainly will get different responses and different guesses of what format you could possibly desire in your output.

Of course, the opposite of wildly random output from the sampling parameter settings of temperature and top-p is that the AI isn’t going to have any memory of prior calls, so if you reduce those to near-zero, you’re going to get the same output, the same questions, every time.

Here is an API Playground preset I created that has the output format more highly-specified, as an example.

https://platform.openai.com/playground/p/3VkD17zUOE8rA1cHkR0StEjD?mode=complete&model=gpt-3.5-turbo-instruct

2 Likes