OpenAI gpt-5-mini with_structured_output returns malformed JSON filled with whitespace instead of valid response

API
,
1

I’m using LangChain’s ChatOpenAI.with_structured_output() with gpt-5-mini to classify user intent into a Pydantic model. Intermittently, the model returns malformed JSON — the response starts with a valid key but never closes the object, instead filling tens of thousands of characters with whitespace/newlines/tabs.

Pydantic schema:

from pydantic import BaseModel, Field
from typing import Literal

class RouteDecision(BaseModel):
    flow: Literal[...] = Field(
        description="Detected intent flow from user message."
    )
    response: str = Field(
        default="",
        description="Direct response text (only for off_topic flow)"
    )

LLM setup:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(temperature=0, model="gpt-5-mini", timeout=15, max_retries=2)
routing_llm = llm.with_structured_output(RouteDecision)

decision = routing_llm.invoke(messages)  # <-- fails here

Error:

pydantic_core._pydantic_core.ValidationError: 1 validation error for RouteDecision
  Invalid JSON: EOF while parsing an object at line 43114 column 0 
  [type=json_invalid, input_value='{"flow":"hmc"  \t\t   \t\...\n \t\t\t\r\n\t\t\t\r\n', input_type=str]

Observations:

  • Happens intermittently, not every call

  • Same issue occurs with gpt-5.4-mini, gpt-5.4-nano, gpt-4-o-mini

  • The schema is simple (2 fields), so it’s not a complexity issue

Environment:

  • langchain-openai==1.1.7

  • pydantic==2.12.5

  • openai==2.15.0

  • Python 3.10.12

Is this a known issue with OpenAI’s structured output? How to fix