Web Search Completion Cuts Off Response and ignores structured outputs on complex prompts

API
,
1

This reminds me of the days before the JSON schema option and JSON Structured outputs.

Currently I have a few complex structured output schemas. These have an extremely high failure rate only with the Web Search Preview model.

The model commonly cuts the content in the middle of the output, breaking the JSON entirely. There is no error or “lack of tokens” etc. It fails silently and breaks with the schema output by ending abruptly mid string. Using the normal GPT4o model does not have this problem.

I have resolved this with some of my requests by vastly simplifying my schema which brought it to a high success right. However not all of my schema outputs can be simplified in such a manner. I am looking forward to a fix or any suggestions if anyone found a way to mitigate this issue

2

First thing I’d check: see the finish_reason that you are getting in the API response. “stop”…or “length”.

This model is particular in that it seems to operate by RAG auto-injection every turn, yet you are not billed by input tokens for that portion (obfuscation of the delivery…). max_tokens isn’t available and shouldn’t be affecting this unusually, which is the first thought that could produce termination.

Then: if you cut off structured outputs in a non-stream response - you’re gonna get an API error if there is enforcement of strict being applied. I would review the “container” between response_format and “schema” to see that you have correct placement of the strict parameter.

Python:

response = client.chat.completions.create(
  model="gpt-4o",
  messages=[],
  response_format={
    "type": "json_schema",
    "json_schema": {
      "name": "lotto_guesses",
      "strict": True,
      "schema": {
        "type": "object",
        "properties": {
          "my_guess": {
            "type": "string",
            "description": "The user's lottery number guess."
          }
        },
        "required": [
          "my_guess"
        ],
        "additionalProperties": False
      }
    }
  },
  ...

You provide no information about your implementation though. You can also build in Python a BaseModel and send that as response_format to the .parse() method, ensuring strict and validation.

You pay every time for search, even for hello, by using this model. In Responses, however, you have a tool, meaning a second iteration, but an AI choice of whether to invoke a search or not (with pattern-damaging injections after).

3

@_j Thank you for the informed response.

The finish reason is indeed “Stop” and not “length” which was my first suspicion when I noticed this behavior. Furthermore I believe strict is correctly applied.

I think this is an openAI bug versus an incorrect usage of their API

But just to be sure, here is a heavily redacted version of my payload

{
  "messages": [
    {
      "role": "system",
      "content": "You are an AI assistant for a software platform. You are assigned to a structured project and must assist the user based on predefined context. Responses must contain plain newlines only—no markdown, no formatting, and no special characters. Respond only in English. Populate 5 items related to the following category: placeholderCategory."
    },
    {
      "role": "user",
      "content": "I want to create the following product: An AI-powered solution designed to enhance a traditional manual process. It adapts to user preferences in real time to provide personalized suggestions. The platform serves both end-users and enterprise clients to improve efficiency and user experience."
    }
  ],
  "model": "gpt-4o-search-preview",
  "response_format": {
    "json_schema": {
      "name": "placeholderSchema",
      "strict": true,
      "schema": {
        "type": "object",
        "required": ["data"],
        "properties": {
          "data": {
            "type": "object",
            "required": ["placeholderCategory"],
            "properties": {
              "placeholderCategory": {
                "description": "Must contain exactly 5 elements",
                "type": "array",
                "items": {
                  "type": "object",
                  "required": ["title", "description", "relatedOrgs"],
                  "properties": {
                    "title": {
                      "type": "string",
                      "description": "Short label for the item"
                    },
                    "description": {
                      "type": "string",
                      "description": "Detailed explanation of the item and suggested next steps"
                    },
                    "relatedOrgs": {
                      "type": "array",
                      "items": {
                        "type": "object",
                        "required": ["name", "url"],
                        "properties": {
                          "name": {
                            "type": "string",
                            "description": "Name of a real, currently active organization"
                          },
                          "url": {
                            "type": "string",
                            "description": "URL of the organization's website"
                          }
                        },
                        "additionalProperties": false
                      }
                    }
                  },
                  "additionalProperties": false
                }
              }
            },
            "additionalProperties": false
          }
        },
        "additionalProperties": false
      }
    }
  },
  "web_search_options": {
    "search_context_size": "high"
  }
}

adding the redacted response

{
  "redacted": "redacted",
  "redacted": "redacted",
  "redacted": "redacted",
  "redacted": [
    {
      "redacted": [
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "redacted"
        },
        {
          "redacted": "redacted",
          "redacted": "Wisconsin

Notice the end mid string, no JSON, no stop reason etc. I believe this to be a bug

4

This is getting worse. Now most requests I make to OpenAI are returned as “successful” and I am charged for it. Yet the Structured Output (with strict set to true) is completely disregarded and I get a JSON object cut off mid string rendering the generation useless and getting charged for it… Not sure what to do, our existing retry system eventually gets it to work but a single generation now takes 10+ tries until OpenAI gets it right

My concern is that since there is no Error on the openAI side, they don’t even know this is happening, and despite submitting a report to the help center, it was only responded to by an obvious ChatGPT bot. Anyone else have a better way to submit a bug report?

5

Did you fix this? Hitting the same error I think…