Web search on Responses API breaks inline citations when passed a pydantic data model as text_format

API Bugs
, ,
4

this issue still exists as of today 28-Dec-25.
**
WebSearchTool with Structured Outputs causes JSON truncation (EOF error) - response grows unbounded**

Environment

  • Model: gpt-4.1-mini-2025-04-14

  • API: Responses API with WebSearchTool

  • Output: Structured Outputs (Pydantic schema)

  • Parameters tested:

  • max_output_tokens=32768 (model max)

  • search_context_size=“low” (minimum)

Problem Description

When using WebSearchTool with structured outputs, the API returns truncated JSON that fails Pydantic validation with:

Invalid JSON: EOF while parsing a list at line X column Y

The response is cut off mid-JSON, producing invalid output regardless of prompt instructions or parameter settings.

Reproduction Steps

  1. Create a Responses API request with:

  • WebSearchTool enabled

  • Structured output schema (Pydantic model)

  • A company/entity search query

  1. Observe truncated JSON in response

Test Results

Attempt Truncation Point Duration Prompt Strategy
1 column 6,791 ~20s Basic
2 column 34,601 ~3.8 min Added max_output_tokens=32768
3 column 48,433 ~4.8 min Added “keep response concise, max 3 items per field”

Key observation: Adding instructions to be concise made the response larger, not smaller. The model appears to ignore output length guidance when web search fills the context.

Workarounds Attempted (None Fully Effective)

  1. :white_check_mark: Set max_output_tokens=32768 - Response grew but still truncated

  2. :white_check_mark: Set search_context_size=“low” - No improvement

  3. :white_check_mark: Added prompt: “Ensure JSON is valid and complete” - No improvement

  4. :white_check_mark: Added prompt: “Maximum 3 items per field, prioritize quality” - Response grew larger

  5. :white_check_mark: Simplified schema to fewer fields - Still truncates

Expected Behavior

  • The API should return valid, parseable JSON that conforms to the schema

  • If context is exhausted, the model should gracefully close the JSON structure

  • search_context_size=“low” should limit internal context consumption

Actual Behavior

  • Web search appears to consume unbounded context internally

  • Model generates partial JSON until some limit is hit

  • Response is truncated mid-structure with no closing brackets

  • No error returned - API reports success with invalid JSON payload

Workaround Implemented

We catch the Pydantic validation error and extract partial data from the truncated response for logging/debugging, but this is not a production-viable solution.

Related Issues

Request

  1. Fix the internal context management for WebSearchTool

  2. If truncation is unavoidable, ensure JSON is properly closed before returning

  3. Return an appropriate error/warning when web search context exceeds limits