I’m trying the new structured output feature but I’m getting a response which is not valid json. I’ve tried the request multiple times and I’m always getting an invalid response.
The full request is:
{
"model" : "gpt-4o-2024-08-06",
"messages" : [
{
"role" : "system",
"content" : "Du bist Teil eines Systems von Agenten, die einen Nutzer im Erbrecht beraten sollen. Deine Aufgabe ist die folgende:\n\nFinde heraus, ob Fine Kinder hat.\n\nGehen zum Lösen der Aufgabe wie folgt vor:\n- Wenn der Nutzer eine Frage stellt, für die du Hintergrundwissen brauchst oder die nichts direkt mit deiner Aufgabe zu tun hat, frage bei der FAQ nach.\n- Wenn der Nutzer sagt, er hat einen Fehler gemacht, sich geirrt, oder dich darum bittet, etwas zu verbessern, frage zu bei der FAQ nach.\n- Wenn du als letztes dran warst und auf eine Antwort des Nutzers warten willst, gebe \"waitForUser\" zurück.\n- Wenn du Fragen stellen oder dem Nutzer etwas mitteilen willst, gib \"userMessage\" zurück.\n- Wenn du deine Aufgabe gelöst hast, gib an die Daten an den nächsten Agenten mit \"nextAgent\" weiter. "
},
{
"role" : "assistant",
"content" : "Wie heißt das Kind von Lotte?"
},
{
"role" : "user",
"content" : "Mae"
},
{
"role" : "assistant",
"content" : "Hat Charlie Kinder?"
},
{
"role" : "user",
"content" : "Ja, eines. Monika."
}
],
"tools" : [
{
"type" : "function",
"function" : {
"name" : "ask_faq",
"description" : "Ask a question to the FAQ",
"parameters" : {
"type" : "object",
"required" : [
"question"
],
"properties" : {
"question" : {
"type" : "string"
}
},
"additionalProperties" : false
}
}
}
],
"temperature" : 0.2,
"response_format" : {
"type" : "json_schema",
"json_schema" : {
"name" : "answer",
"description" : "The answer to the task",
"strict" : true,
"schema" : {
"type" : "object",
"required" : [
"answer"
],
"properties" : {
"answer" : {
"anyOf" : [
{
"type" : "object",
"properties" : {
"kind" : {
"type" : "string",
"enum" : [
"nextAgent"
]
},
"data" : {
"type" : "object",
"required" : [
"children"
],
"properties" : {
"children" : {
"items" : {
"type" : "string"
},
"description" : "\nThe names of the children if there were any and they were named.\nNull if the user said there are no children.\nEmpty if there are children, but they were not named.\n",
"type" : [
"array",
"null"
]
}
},
"additionalProperties" : false
}
},
"required" : [
"kind",
"data"
],
"additionalProperties" : false,
"description" : "You have solved your goal and gathered the data and want to hand over to the next agent. If you want to tell the user something, do it before this."
},
{
"type" : "object",
"properties" : {
"kind" : {
"type" : "string",
"enum" : [
"messageUser"
]
},
"message" : {
"type" : "string"
}
},
"required" : [
"kind",
"message"
],
"additionalProperties" : false,
"description" : "You want to ask the user something or tell the user something"
},
{
"type" : "object",
"properties" : {
"kind" : {
"type" : "string",
"enum" : [
"waitForUser"
]
}
},
"required" : [
"kind"
],
"additionalProperties" : false,
"description" : "You need to wait for the user to respond"
}
]
}
},
"additionalProperties" : false
}
}
}
},
and the response was
{
"choices" : [
{
"index" : 0,
"message" : {
"role" : "assistant",
"content" : "{\"answer\":{\"kind\":\"messageUser\",\"message\":\"Hat Fine Kinder?\"}}\n{\"answer\":{\"kind\":\"waitForUser\"}}"
}
}
]
}
So the response contains two different JSON objects, separated by a new line, which is itself not valid JSON.
Am I doing something wrong here? I thought the structured outputs would never generate invalid JSON? Maybe I’m missing an attribute somewhere? Any hints?