Structured Outputs with Assistants

API
1

Has anyone got the new Structured Responses working with Assistants?

I keep getting ‘Object of type ModelMetaclass is not JSON serializable’

eg.

from pydantic import BaseModel

class Insights(BaseModel):
        segments: List[str]
        garm: List[str]
        reasoning: str

run = await openai_client.beta.threads.runs.create(
        model="gpt-4o-2024-08-06",
        thread_id=thread_id,
        assistant_id='',
        temperature=0,
        response_format=Insights
    )

Am I doing something wrong? I am using the Async client too.

2

Following the example code on the documentation page I also could not get the code to run - was getting the same exception.

Try writing it like the code below to fix it:

         response_format={
           'type': 'json_schema',
           'json_schema': 
              {
                "name":"whocares", 
                "schema": Insights.model_json_schema()
              }
         }
5 Likes
3

Thanks, they really need to add this to the Documentation.

4

Firstly, thank you for this. It really does need to be added to the documentation. The docs are not clear at all.

It should be noted that all tools must be of type function when response_format is of type json_schema.

This is/was in the documentation in some form but it is not made clear. The documentation at https://platform.openai.com/docs/guides/structured-outputs/introduction makes it sound like you can use it When using function calling or When using a json_schema response format but that does not apply to assistants.

I tried implementing json_schema with tools=[{"type": "file_search"}] and it is still not a thing. I’ve needed this feature so it’s more like chat completions for a while but right now; I’m having to convince the assistant API to respond with JSON through some kludgery instead, and then dealing with broken JSON output through exception handling.

Ugghhh.

1 Like
5

i spent so much time trying to troubleshoot this. thank you for creating this topic.

1 Like
6

I’m dealing with the same problem - not being able to use the tools. One workaround is to do two consecutive runs for a given prompt - first one to solve the problem using whatever the tools you need to use. The second one with the structured output and the prompt “please output the results as json” - the results from the first run will be in the immediate message history so should work ok.

7

I am still having issues with setting the assistant with the structured outputs.
First, it’s unclear where we should set it(nothing was mentioned in the documentation). in the creation of the assistant/messages/threads?
secondly, I tried out the suggestion above as follows:

    class Step(BaseModel):
        explanation: str
        output: str

    
    class ReportReasoning(BaseModel):
        steps: list[Step]
        final_answer: str

        new_run = client.beta.threads.runs.create(
            thread_id=new_thread.id,
            assistant_id=assistant.id,
            response_format={
                'type': 'json_schema',
                'json_schema':
                    {
                        "name": "whocares",
                        "schema": ReportReasoning.model_json_schema()
                    }},
        )

THIS causes the next error -

openai.BadRequestError: Error code: 400 - {'error': {'message': 'Invalid tools: all tools must be of type `function` when `response_format` is of type `json_schema`.', 'type': 'invalid_request_error', 'param': 'response_format', 'code': None}}
8

The answer is in the error message. You can use any other tools than your own functions. So no code interpreter, no file search. you can set tools = in the call to create the run - this will override whatever tools you have set up on the agent. This way you can at least check, that the structured output part is working. To use tools split your workflow into two parts - create a run where you try to solve the task given by the prompt with all the tools you need. In the second run disable all the tools but set the structured output schema and in the prompts ask the agent to “please output the results as the given json” or something of the like.