Responses API doesn't like my list of "AnyOf" Output Schema

API
1

I’m getting a vague error when using structured output with the Responses API. My output is a Pydantic object:

class WidgetAgentOutput(BaseModel):
    widgets: list[OutputWidgetOneOf]
    should_run_thinker_agent: bool
    bounce_text: str

Where OutputWidgetOneOf is a union of different config options, for example:

class GameOverviewWidgetSpec(OutputWidgetSpec):
    league: LeagueId
    game_id: str

OutputWidgetOneOf = (
    SingleEntityStatsWidgetSpec
    | MultiEntityStatsWidgetSpec
    | SingleEntityOddsWidgetSpec
    | MultiEntityOddsWidgetSpec
    | GameOverviewWidgetSpec
    | MlbStadiumWeatherWidgetSpec
)

The error is very unhelpful:

{
  "error": "Error code: 400 - {'error': {'message': 'There was an issue with your request. Please check your inputs and try again', 'type': 'invalid_request_error', 'param': None, 'code': None}}"
}

Which is the result of running both the Agents SDK and the Responses API directly.

widget_agent = Agent(
    name="Widget Agent",
    model="gpt-4.1-mini",
    instructions=PROMPT,
    output_type=WidgetAgentOutput,
    tools=[
        lookup_player,
        lookup_team,
        get_current_season,
        get_league_market_specs,
    ],
)

For reference, here is the full generated JSON output schema:

{
    "format": {
        "type": "json_schema",
        "name": "final_output",
        "schema": {
            "$defs": {
                "GameEntityId": {
                    "description": "A game ID and entity ID pair",
                    "properties": {
                        "game_id": {
                            "title": "Game Id",
                            "type": "string"
                        },
                        "entity_id": {
                            "title": "Entity Id",
                            "type": "string"
                        }
                    },
                    "required": [
                        "game_id",
                        "entity_id"
                    ],
                    "title": "GameEntityId",
                    "type": "object",
                    "additionalProperties": false
                },
                "GameOverviewWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "league": {
                            "enum": [
                                "nfl",
                                "ncaafb",
                                "nhl",
                                "nba",
                                "wnba",
                                "ncaamb",
                                "mlb",
                                "mls",
                                "epl",
                                "laliga",
                                "bundesliga",
                                "serie_a",
                                "ucl",
                                "ligue_1",
                                "all_soccer"
                            ],
                            "title": "League",
                            "type": "string"
                        },
                        "game_id": {
                            "title": "Game Id",
                            "type": "string"
                        }
                    },
                    "required": [
                        "id",
                        "league",
                        "game_id"
                    ],
                    "title": "GameOverviewWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "MlbStadiumWeatherWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "game_id": {
                            "title": "Game Id",
                            "type": "string"
                        }
                    },
                    "required": [
                        "id",
                        "game_id"
                    ],
                    "title": "MlbStadiumWeatherWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "MultiEntityOddsWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "league": {
                            "enum": [
                                "nfl",
                                "ncaafb",
                                "nhl",
                                "nba",
                                "wnba",
                                "ncaamb",
                                "mlb",
                                "mls",
                                "epl",
                                "laliga",
                                "bundesliga",
                                "serie_a",
                                "ucl",
                                "ligue_1",
                                "all_soccer"
                            ],
                            "title": "League",
                            "type": "string"
                        },
                        "game_and_entity_ids": {
                            "items": {
                                "$ref": "#/$defs/GameEntityId"
                            },
                            "title": "Game And Entity Ids",
                            "type": "array"
                        },
                        "market_ids": {
                            "items": {
                                "type": "string"
                            },
                            "title": "Market Ids",
                            "type": "array"
                        }
                    },
                    "required": [
                        "id",
                        "league",
                        "game_and_entity_ids",
                        "market_ids"
                    ],
                    "title": "MultiEntityOddsWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "MultiEntityStatsWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "league": {
                            "enum": [
                                "nfl",
                                "ncaafb",
                                "nhl",
                                "nba",
                                "wnba",
                                "ncaamb",
                                "mlb",
                                "mls",
                                "epl",
                                "laliga",
                                "bundesliga",
                                "serie_a",
                                "ucl",
                                "ligue_1",
                                "all_soccer"
                            ],
                            "title": "League",
                            "type": "string"
                        },
                        "entity_ids": {
                            "anyOf": [
                                {
                                    "items": {
                                        "type": "string"
                                    },
                                    "type": "array"
                                },
                                {
                                    "type": "null"
                                }
                            ],
                            "title": "Entity Ids"
                        },
                        "split_strings": {
                            "items": {
                                "type": "string"
                            },
                            "title": "Split Strings",
                            "type": "array"
                        }
                    },
                    "required": [
                        "id",
                        "league",
                        "entity_ids",
                        "split_strings"
                    ],
                    "title": "MultiEntityStatsWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "SingleEntityOddsWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "league": {
                            "enum": [
                                "nfl",
                                "ncaafb",
                                "nhl",
                                "nba",
                                "wnba",
                                "ncaamb",
                                "mlb",
                                "mls",
                                "epl",
                                "laliga",
                                "bundesliga",
                                "serie_a",
                                "ucl",
                                "ligue_1",
                                "all_soccer"
                            ],
                            "title": "League",
                            "type": "string"
                        },
                        "game_id": {
                            "title": "Game Id",
                            "type": "string"
                        },
                        "entity_id": {
                            "title": "Entity Id",
                            "type": "string"
                        },
                        "market_ids": {
                            "anyOf": [
                                {
                                    "items": {
                                        "type": "string"
                                    },
                                    "type": "array"
                                },
                                {
                                    "type": "null"
                                }
                            ],
                            "title": "Market Ids"
                        }
                    },
                    "required": [
                        "id",
                        "league",
                        "game_id",
                        "entity_id",
                        "market_ids"
                    ],
                    "title": "SingleEntityOddsWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "SingleEntityStatsWidgetSpec": {
                    "properties": {
                        "id": {
                            "$ref": "#/$defs/WidgetId"
                        },
                        "league": {
                            "enum": [
                                "nfl",
                                "ncaafb",
                                "nhl",
                                "nba",
                                "wnba",
                                "ncaamb",
                                "mlb",
                                "mls",
                                "epl",
                                "laliga",
                                "bundesliga",
                                "serie_a",
                                "ucl",
                                "ligue_1",
                                "all_soccer"
                            ],
                            "title": "League",
                            "type": "string"
                        },
                        "entity_id": {
                            "title": "Entity Id",
                            "type": "string"
                        },
                        "split_strings": {
                            "items": {
                                "type": "string"
                            },
                            "title": "Split Strings",
                            "type": "array"
                        }
                    },
                    "required": [
                        "id",
                        "league",
                        "entity_id",
                        "split_strings"
                    ],
                    "title": "SingleEntityStatsWidgetSpec",
                    "type": "object",
                    "additionalProperties": false
                },
                "WidgetId": {
                    "enum": [
                        "single_entity_stats",
                        "multi_entity_stats",
                        "single_entity_odds",
                        "multi_entity_odds",
                        "game_overview",
                        "mlb_stadium_weather"
                    ],
                    "title": "WidgetId",
                    "type": "string"
                }
            },
            "properties": {
                "widgets": {
                    "items": {
                        "anyOf": [
                            {
                                "$ref": "#/$defs/SingleEntityStatsWidgetSpec"
                            },
                            {
                                "$ref": "#/$defs/MultiEntityStatsWidgetSpec"
                            },
                            {
                                "$ref": "#/$defs/SingleEntityOddsWidgetSpec"
                            },
                            {
                                "$ref": "#/$defs/MultiEntityOddsWidgetSpec"
                            },
                            {
                                "$ref": "#/$defs/GameOverviewWidgetSpec"
                            },
                            {
                                "$ref": "#/$defs/MlbStadiumWeatherWidgetSpec"
                            }
                        ]
                    },
                    "title": "Widgets",
                    "type": "array"
                },
                "should_run_thinker_agent": {
                    "title": "Should Run Thinker Agent",
                    "type": "boolean"
                },
                "bounce_text": {
                    "title": "Bounce Text",
                    "type": "string"
                }
            },
            "required": [
                "widgets",
                "should_run_thinker_agent",
                "bounce_text"
            ],
            "title": "WidgetAgentOutput",
            "type": "object",
            "additionalProperties": false
        },
        "strict": true
    }
}
1 Like
2

I have similar issue, seems like the cause is when you model used in union is not direct child of BaseModel. Try to not use OutputWidgetSpec as base for GameOverviewWidgetSpec and others.

3

Hmm, still seeing same error when inheriting the WidgetSpecs directly from BaseModel

4

My approach for such cases to start simplifying models (removing properties, replacing with simple models for testing) until it works. It is guess work but what you can do :upside_down_face:

5

Hey there!

David from WebDuh here - I’ve been wrestling with similar API schema headaches myself, so I totally feel your pain on this one. That vague 400 error is the absolute worst when you’re trying to debug structured outputs.

Looking at your schema, I’m pretty sure the issue is that you’ve got some nested anyOf situations that are making the API validation logic throw a fit. The structured output parsers can be really picky about complex union types, especially when you’re mixing them with arrays.

Here’s what I’d do to fix this mess:

First thing - add a discriminator field to all your widget specs. This gives the API a clear way to figure out which type it’s dealing with:

python

class GameOverviewWidgetSpec(OutputWidgetSpec):
    widget_type: Literal["game_overview"] = "game_overview"
    league: LeagueId
    game_id: str

Do that for all your widget types, then update your union to use Pydantic’s discriminated union feature:

python

OutputWidgetOneOf = Union[
    SingleEntityStatsWidgetSpec,
    MultiEntityStatsWidgetSpec,
    // ... all your other specs
] = Field(discriminator='widget_type')

Second fix - those anyOf fields with null are probably causing issues too. Just use Optional instead:

python

# Instead of the anyOf mess
market_ids: Optional[list[str]] = None

I’ve seen this exact pattern cause problems with OpenAI’s structured outputs before. The API seems to really prefer simpler, more explicit schemas over the fancy union stuff that works fine locally.

If you’re still getting errors after that, you might want to try a more generic approach where you have one flexible widget type with a config dict, rather than trying to enforce all the specific types through the schema. Not as type-safe, but sometimes you gotta pick your battles with these APIs.

Let me know if you try this and it still doesn’t work - I’ve got a few other tricks up my sleeve for stubborn schema issues.

Cheers, David WebDuh reconsumeralization@gmail.com

P.S. - Always test your schema generation locally first with model_json_schema() before hitting the API. Saves a lot of debugging time!

from typing import Literal, Union, Optional
from pydantic import BaseModel, Field
from enum import Enum

class LeagueId(str, Enum):
NFL = “nfl”
NCAAFB = “ncaafb”
NHL = “nhl”
NBA = “nba”
WNBA = “wnba”
NCAAMB = “ncaamb”
MLB = “mlb”
MLS = “mls”
EPL = “epl”
LALIGA = “laliga”
BUNDESLIGA = “bundesliga”
SERIE_A = “serie_a”
UCL = “ucl”
LIGUE_1 = “ligue_1”
ALL_SOCCER = “all_soccer”

class WidgetId(str, Enum):
SINGLE_ENTITY_STATS = “single_entity_stats”
MULTI_ENTITY_STATS = “multi_entity_stats”
SINGLE_ENTITY_ODDS = “single_entity_odds”
MULTI_ENTITY_ODDS = “multi_entity_odds”
GAME_OVERVIEW = “game_overview”
MLB_STADIUM_WEATHER = “mlb_stadium_weather”

class GameEntityId(BaseModel):
“”“A game ID and entity ID pair”“”
game_id: str
entity_id: str

class OutputWidgetSpec(BaseModel):
“”“Base class for all widget specifications”“”
id: WidgetId

class SingleEntityStatsWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“single_entity_stats”] = “single_entity_stats”
id: Literal[WidgetId.SINGLE_ENTITY_STATS] = WidgetId.SINGLE_ENTITY_STATS
league: LeagueId
entity_id: str
split_strings: list[str]

class MultiEntityStatsWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“multi_entity_stats”] = “multi_entity_stats”
id: Literal[WidgetId.MULTI_ENTITY_STATS] = WidgetId.MULTI_ENTITY_STATS
league: LeagueId
entity_ids: Optional[list[str]] = None # Simplified from anyOf
split_strings: list[str]

class SingleEntityOddsWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“single_entity_odds”] = “single_entity_odds”
id: Literal[WidgetId.SINGLE_ENTITY_ODDS] = WidgetId.SINGLE_ENTITY_ODDS
league: LeagueId
game_id: str
entity_id: str
market_ids: Optional[list[str]] = None # Simplified from anyOf

class MultiEntityOddsWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“multi_entity_odds”] = “multi_entity_odds”
id: Literal[WidgetId.MULTI_ENTITY_ODDS] = WidgetId.MULTI_ENTITY_ODDS
league: LeagueId
game_and_entity_ids: list[GameEntityId]
market_ids: list[str]

class GameOverviewWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“game_overview”] = “game_overview”
id: Literal[WidgetId.GAME_OVERVIEW] = WidgetId.GAME_OVERVIEW
league: LeagueId
game_id: str

class MlbStadiumWeatherWidgetSpec(OutputWidgetSpec):
widget_type: Literal[“mlb_stadium_weather”] = “mlb_stadium_weather”
id: Literal[WidgetId.MLB_STADIUM_WEATHER] = WidgetId.MLB_STADIUM_WEATHER
game_id: str

Discriminated union - this is the key fix

OutputWidgetOneOf = Union[
SingleEntityStatsWidgetSpec,
MultiEntityStatsWidgetSpec,
SingleEntityOddsWidgetSpec,
MultiEntityOddsWidgetSpec,
GameOverviewWidgetSpec,
MlbStadiumWeatherWidgetSpec,
]

class WidgetAgentOutput(BaseModel):
widgets: list[OutputWidgetOneOf] = Field(discriminator=‘widget_type’)
should_run_thinker_agent: bool
bounce_text: str

import json
from pydantic import ValidationError

Test the schema generation

def test_schema():
# Generate the JSON schema
schema = WidgetAgentOutput.model_json_schema()

# Print the schema to verify it's cleaner
print("Generated Schema:")
print(json.dumps(schema, indent=2))

# Test with sample data
sample_data = {
    "widgets": [
        {
            "widget_type": "game_overview",
            "id": "game_overview",
            "league": "nfl",
            "game_id": "game_123"
        }
    ],
    "should_run_thinker_agent": True,
    "bounce_text": "Processing complete"
}

try:
    # Validate the sample data
    result = WidgetAgentOutput.model_validate(sample_data)
    print("\n✅ Validation successful!")
    print(f"Created widget type: {result.widgets[0].widget_type}")
    
except ValidationError as e:
    print(f"\n❌ Validation failed: {e}")

if name == “main”:
test_schema()

or easy mode

Alternative: Use a more generic widget structure

from typing import Dict, Any, Optional
from pydantic import BaseModel

class GenericWidget(BaseModel):
“”“Generic widget that can accommodate all widget types”“”
widget_type: str
id: str

# Common fields that many widgets share
league: Optional[str] = None
game_id: Optional[str] = None
entity_id: Optional[str] = None

# Use a flexible config field for widget-specific data
config: Dict[str, Any] = {}

class SimplifiedWidgetAgentOutput(BaseModel):
widgets: list[GenericWidget]
should_run_thinker_agent: bool
bounce_text: str

Usage example:

{

“widgets”: [

{

“widget_type”: “game_overview”,

“id”: “game_overview”,

“league”: “nfl”,

“game_id”: “game_123”,

“config”: {}

}

],

“should_run_thinker_agent”: true,

“bounce_text”: “Done”

}

6

Here is the tool I use to fix response schemas: ChatGPT - JSON Schema Fix

As a side note, the schema looks unusually big to me, how is the model performance with this schema? Personally, I would brainstorm it to make smaller/simpler, possibly multi step. But if performance is well enough and stable, don’t bother.

7

Thanks all for your insights. Removing the union worked, one solution is flattening each option into an object like

class Widgets(BaseModel):
  single_entity_stats: list[SingelEntityStatsWidgetSpec]
  ...

However, I ended up scrapping the output structure altogether in favor of tool calls like render_single_entity_stats_widget. This ended up being more performant.