Nested pydantic model with structured outputs

[jonathan.bouchet]

Hello,
I would like to know if nested pydantic models are supported by the newly structured outputs (which is super useful, thanks)

I tried the following:

class RouterEval(BaseModel):
    topic: str = Field(description="the topic of the query provided by the human")
    query: str = Field(description="the statement of the user")

class Topics(StrEnum):
    politic="politic"
    sport="sport"
    other="other"
    mathematics="mathematics"

class RouterEvalWithEnum(BaseModel):
    topic: Topics = Field(description="the topic of the query provided by the human")
    query: str = Field(description="the statement of the user")

Tests:

queries: list = [
    "who won the superbowl in 2023 ?",
    "who was the first president of the United States",
    "what is 1+1 equal to ?",
    "what is the weather in Paris ?"
]

res = []
for q in queries:

    completion = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": "Classify the topic of the query between `politic`, `sport`, `mathematics` and `other`. Use `other` if not related to politic, sport, nor mathematics"},
            {"role": "user", "content": q},
        ],
        response_format=RouterEval,
        temperature=0.0
    )
    res.append(completion.choices[0].message.parsed)


# RouterEval(topic='sport', query='who won the superbowl in 2023 ?'),
# RouterEval(topic='politic', query='who was the first president of the United States'),
# RouterEval(topic='mathematics', query='what is 1+1 equal to ?'),
# RouterEval(topic='other', query='what is the weather in Paris ?')

But when trying with the model with enum, I got the following error:

BadRequestError: Error code: 400 - {‘error’: {‘message’: “Invalid schema for response_format ‘RouterEvalWithEnum’: In context=(‘properties’, ‘topic’), ‘allOf’ is not permitted”, ‘type’: ‘invalid_request_error’, ‘param’: ‘response_format’, ‘code’: None}}

My env.

• Python 3.11.8 (main, Mar 24 2024, 00:21:05) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin
• openai 1.40.1
• pydantic 2.8.2
• pydantic_core 2.20.1

Regards,
Jonathan

[jonathan.bouchet]

interesting.
I changed topic to be a list of Topics instead a single one and it went through

class RouterEvalWithEnum(BaseModel):
    topic: list[Topics] = Field(description="the topic of the query provided by the human")
    query: str = Field(description="the statement of the user")

the response is now a list of Topics, so you just need to loop through it.

queries: list = [
    "who won the superbowl in 2023 ?",
    "who was the first president of the United States",
    "what is 1+1 equal to ?",
    "what is the weather in Paris ?"
]

res = []
for q in queries:

    completion = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": "Classify the topic of the query between `politic`, `sport`, `mathematics` and `other`. Use `other` if not related to politic, sport, nor mathematics"},
            {"role": "user", "content": q},
        ],
        response_format=RouterEvalWithEnum,
        temperature=0.0
    )
    res.append(completion.choices[0].message.parsed)

# RouterEvalWithEnum(topic=[<Topics.sport: 'sport'>], query='who won the superbowl in 2023 ?'),
# RouterEvalWithEnum(topic=[<Topics.politic: 'politic'>], query='who was the first president of the United States'),
# RouterEvalWithEnum(topic=[<Topics.mathematics: 'mathematics'>], query='what is 1+1 equal to ?'),
# RouterEvalWithEnum(topic=[<Topics.other: 'other'>], query='what is the weather in Paris ?')

[ganatradarpan]

Did you end up figuring out why this was a problem?

EDIT: Ignore this question, I saw there was an answer in the post named: Structured Outputs Error – allOf error (apologies I cannot add a link)

[expertise.ai.chat]

@jonathan.bouchet I actually had the same issue, which I documented here.

Structured Output works for Flat Schema, not for Nested Schema

I am using structured outputs to ensure GPT’s response conforms to a schema I define. I have defined this schema using JSON. Here is my Schema. class ContentsSchema(BaseModel): contents: List[ContentSchema] = Field(..., description=<some_description>) summary: str = Field(..., description=<some_description>) As you can see, this schema depends on a few other schemas. I will define them below. class ContentSchema(BaseModel): number: str = Field(..., description=<some_description>)…

If you really don’t want to use a list, I’ve found another admittedly dumb solution. Try changing the name of your Topics class. I don’t know why, but I have consistently found this fixes the bug you’re experiencing. You may have to try a few different options, but I’m confident it will work.