JSON Response format with assistant runs

API
1

Is it possible to set the response_format as json when using the assistant api instead of the classic chat api?

Doesn’t seem to be a way to do it in the docs, but seems silly since you can set files, tools, system messages, and more at the assistant, thread, and message level when using assistant api.

6 Likes
2

Afraid not, as of November 2023, JSON mode is not supported in the Assistants API

3

I have noticed that the instruction following capability of the Assistant is poor. I have requested it to return the answer in JSON format, but it still outputs some other information.

1 Like
4

Ability to support Assistants API with JSON mode would be helpful.

Noticed that assistant quite often stops returning response as JSON.

Are the any plans in near future to enable this functionality?

1 Like
5

This really limits the usefulness of the assistant API in situations where I need to be able to inject structured data. Are there any plans to add this?

1 Like
6

It surprises me that it is not already supported. I hope that it will be added in the near future.

1 Like
7

@robmartin You can take the response from the Assistant and then run it through the chat api to have it format it in JSON for you. Should work for the time being at least until they add it into the Assistant API.

1 Like
8

I agree, not being able to get the data the same way every time is nearly impossible to code reliably to.

9

I don’t think this is exactly what you’re looking for, but I thought I’d share the function I’m using to print the responses from Assistant API calls:

def obj_dict(obj):
    return obj.__dict__

json_string = json.dumps(messages_list, default=obj_dict, indent=4)

This example is for messages_list, a list of messages from a thread, but should work for any response.

10

Give the model only one tool:

{
    'name': 'respond',
    'description': 'Respond to the user in a structured manner.',
    'parameters': {
        'properties': {
            'structured_response': {
                'type': 'object',
                'description': 'Your structured response. Only accepts JSON format!'
            }
        },
        'required': ['structured_response'],
        'type': 'object'
    }
}

And tell the asisstant in the instructions to always use the this tool:
You are a personal assistant. You always use the respond tool. You can’t talk to the user directly. You can only use the respond tool to communicate with the user. You have to call the respond tool. You only output structured json objects.

This works 99% of the time for me.

6 Likes
11

Wish I could click the Like button 1000 times.

3 Likes
12

I am unfortunately unable to get this working, as my assistant instruction are about 6 to 7 and my guess is that it is getting lost in there. BTW I am trying this out in the playground and many times the assistant doesn’t even call the respond tool. Anyone else going through the same?

13

I was able to solve this with very specific instructions that now are reliable.

At the end of my instructions of one of my JSON generation assistants:

Response format instructions:
The response must be a single JSON object with one attribute ‘Summary’ that has your detailed summary containing the full text of your work. There will be no text or other characters outside the JSON string.

14

Thanks for example. In case anybody else has formatting issues when adding this via the Assistants GUI, I had to replace the ’ with " otherwise it would not allow me to create the new Function.

15

I have it working also with custom instructions. It has not failed yet. To avoid text it may want to put for clarification or ambiguates, I also instruct to put in JSON. Here are my instructions.

Incomplete Instruction Handling: Autonomously fills in details and provides the best arrangement of choices, explaining ambiguities with ‘CLARIFY’ in the JSON key of ambiguities.
JSON Output: Generates choices in a structured JSON format, including: token estimate, title, tags, level with this structure:

{
title: string,
tags: [string]
level: string
ambiguities: [string]
estimated_token_usage: number

}

And array of choices, with this structure:

{
choices: [
{
choice_text: string
choice_type: string
}
]
}

16

It will often output the json as markdown ```json

However, I just run it through a parser and validate JSON before further processing.

17

Using a tool is an interesting idea but I think it’s uneccesary for anyone who wants JSON output only.

I’m having good results with just using this in the user role text.

<The Setup Instructions>

The output structure must be a valid JSON object with a structure like:
<YOUR JSON STRUCTURE>

<The Actual Task to Perform>

JSON output with no extraneous text or wrappers:

This prompts ChatGPT to simply output JSON data suitable for json.load() in response which eliminates the following kind of output every time (at least for me so far):

   ```json
   {structured stuff}
    ```
1 Like