Structured output is not structured

[nlar]

Hi,

I am a new to using the API and is trying to use the Structured Output, but it is causing me some some issues, because the output I get is not structured.

I have used the example from the Introducing Structured Outputs in the API | OpenAI.

The request looks like this:

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful math tutor."
    },
    {
      "role": "user",
      "content": "solve 8x + 31 = 2"
    }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "math_response",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "steps": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "explanation": {
                  "type": "string"
                },
                "output": {
                  "type": "string"
                }
              },
              "required": ["explanation", "output"],
              "additionalProperties": false
            }
          },
          "final_answer": {
            "type": "string"
          }
        },
        "required": ["steps", "final_answer"],
        "additionalProperties": false
      }
    }
  }
}

According to the article, the output should be similar to this:

{
  "steps": [
    {
      "explanation": "Subtract 31 from both sides to isolate the term with x.",
      "output": "8x + 31 - 31 = 2 - 31"
    },
    {
      "explanation": "This simplifies to 8x = -29.",
      "output": "8x = -29"
    },
    {
      "explanation": "Divide both sides by 8 to solve for x.",
      "output": "x = -29 / 8"
    }
  ],
  "final_answer": "x = -29 / 8"
}

But the output I am getting is not structured. Everything is in the message/content part of the JSON.

I have tried with both gpt-4o-mini-2024-07-18 and gpt-4o-2024-08-06 but the results are the same.

This is what I get:

{
    "choices": [
        {
            "content_filter_results": {
                "hate": {
                    "filtered": false,
                    "severity": "safe"
                },
                "protected_material_code": {
                    "filtered": false,
                    "detected": false
                },
                "protected_material_text": {
                    "filtered": false,
                    "detected": false
                },
                "self_harm": {
                    "filtered": false,
                    "severity": "safe"
                },
                "sexual": {
                    "filtered": false,
                    "severity": "safe"
                },
                "violence": {
                    "filtered": false,
                    "severity": "safe"
                }
            },
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "message": {
                "content": "{\"steps\":[{\"explanation\":\"Subtract 31 from both sides to isolate the term with x.\",\"output\":\"8x + 31 - 31 = 2 - 31\"},{\"explanation\":\"This simplifies to 8x = -29.\",\"output\":\"8x = -29\"},{\"explanation\":\"Now, divide both sides by 8 to solve for x.\",\"output\":\"x = -29 / 8\"},{\"explanation\":\"Simplifying -29 / 8 gives the final answer.\",\"output\":\"x = -3.625\"}],\"final_answer\":\"x = -3.625\"}",
                "refusal": null,
                "role": "assistant"
            }
        }
    ],
    "created": 1739881035,
    "id": "XXX",
    "model": "gpt-4o-mini-2024-07-18",
    "object": "chat.completion",
    "prompt_filter_results": [
        {
            "prompt_index": 0,
            "content_filter_results": {
                "hate": {
                    "filtered": false,
                    "severity": "safe"
                },
                "jailbreak": {
                    "filtered": false,
                    "detected": false
                },
                "self_harm": {
                    "filtered": false,
                    "severity": "safe"
                },
                "sexual": {
                    "filtered": false,
                    "severity": "safe"
                },
                "violence": {
                    "filtered": false,
                    "severity": "safe"
                }
            }
        }
    ],
    "system_fingerprint": "XXX",
    "usage": {
        "completion_tokens": 119,
        "completion_tokens_details": {
            "accepted_prediction_tokens": 0,
            "audio_tokens": 0,
            "reasoning_tokens": 0,
            "rejected_prediction_tokens": 0
        },
        "prompt_tokens": 85,
        "prompt_tokens_details": {
            "audio_tokens": 0,
            "cached_tokens": 0
        },
        "total_tokens": 204
    }
}

I am testing through Postman and via Azure AI Foundry.

Am I doing something wrong - or does structured outputs have to be enabled somehow?

[j.wischnat]

Welcome to the community!
While I don’t use structured outputs, I can’t say for sure, but in the example you provided, it looks pretty structured to me, no?
It looks like in the message content key it contains a valid json, right?
I’d imagine you just convert that to a json and you should be fine!
Is there something else you are expecting?

Cheers!

[_j]

I would have to conclude by not including your deployment id name as model parameter in an API call, that you’ve directed your request to some moderation filter for AI that checks the inputs and returns flagged content. See the tags naming Azure’s “Protected Material - text” and “Protected Material - code” that detects recitation of copyrighted material.

[nlar]

You might be right.

I just found the documentation from Microsoft, and the output example they have provided look similar to what I am getting.

I was expecting an actual JSON output like the one in the example.

Moreover, I was trying to categorize data, where the model should only use pre-defined categories. That wasn’t working as well - so I thought I might did something wrong.

I’ll keep testing a bit more.

{
  "name": "my_schema",
  "schema": {
    "type": "object",
    "properties": {
      "category": {
        "type": "string",
        "description": "The category of the content.",
        "enum": [
          "Business",
          "Politics",
          "Sport",
          "Entertainment",
          "Science",
          "Art"
        ]
      },
      "text": {
        "type": "string",
        "description": "The text providing details or context."
      },
      "page": {
        "type": "number",
        "description": "The page number associated with the content."
      },
      "reasoning": {
        "type": "string",
        "description": "The reasoning or context for the content in this section."
      }
    },
    "required": [
      "category",
      "text",
      "page",
      "reasoning"
    ],
    "additionalProperties": false
  },
  "strict": true
}

[nlar]

Is the deployment id name this from the example?

[_j]

Have you paid credits to platform.openai.com, or are you using Azure OpenAI services by Microsoft?
Are you showing a side-channel JSON for moderations that are part of running on an AI hosting sites services?

The POST URL has to be made to the domain api.openai.com, otherwise you are just requesting a local URL.

Obviously something is going quite wrong if you are getting the flagging results of submitting to a moderations (content filtering) engine instead of the intended AI outpu.

With OpenAI, you pick the existing model name, while with Azure, you must create a deployment of a model in a region, which you can give a name similar to the model.

[nlar]

@j.wischnat

I’d imagine you just convert that to a json and you should be fine!

You were right. I could simply convert that into JSON and then it works for me. Thanks for the help.