Handling Structured Output in Function Tool Calls with `file_search`

Community
,
1

I’m using OpenAI assistants with the file_search tool and leveraging function calls to get structured output from the model.

When I receive structured data in the function tool call arguments, I’m unsure what to return in the submit tool call response. Currently, I’ve tried:

  1. Returning "true"
  2. Returning the same structured output that was received in the function call

However, in both cases, I still see an additional Assistant message in the thread, which is part of the structured output. This behavior makes me unsure of the correct way to handle structured responses in function tool calls.

Should I be handling this differently to suppress the extra Assistant message? Or is this the expected behavior? Any guidance on best practices for structured outputs in function tool calls would be greatly appreciated!

2

If you want a structured output, you would use the parameter response_format when you create an assistant - a response format JSON schema.

Functions always work as an external tool you provide that will return results the AI can report on. They are not for generating a final output.

1 Like
3

I see. The main difference was to develop an assistant using OpenAI’s API. The user interface provided for creating assistants does not allow the combination of file search functionality with different response formats. However, I was able to use the API to successfully create an assistant that incorporates both a specific response format and a file search tool.

Despite this, the UI continues to show the response format as text.

4

Runs still Failing though. Just says “something went wrong”. No incomplete reasons. Nothing descriptive in run object.

Threads and Assistants seems to be such a broken experience :slightly_frowning_face:

5

Indeed yes, OpenAI are too clever to have have something that works on chat completions be functional in assistants.

However, you can just prompt for the structured output and get your JSON response, just powered by the intelligence:

The image shows a screenshot of a QA exchange about using OpenAI's Assistants endpoint with a JSON response format and file search feature, including a JSON response suggesting consulting the official documentation for implementation details. (Captioned by AI)
The image shows a screenshot of a QA exchange about using OpenAI's Assistants endpoint with a JSON response format and file search feature, including a JSON response suggesting consulting the official documentation for implementation details. (Captioned by AI)540×382 10.8 KB

This success in following a schema was written very similar to how the actual structured schema is placed at the end of the first system message when using “response_format”:


# Response Formats

## mandatory JSON output using schema

{
    "type": "object",
    "properties": {
      "answer": {
        "type": "string",
        "description": "The answer derived from the query."
      },
      "search_result": {
        "type": "boolean",
        "description": "Indicates if the answer was obtained from file search information."
      }
    },
    "required": [
      "answer",
      "search_result"
    ],
    "additionalProperties": false
  }
6

Ya i have seen this suggestion in other posts on this community. But this is not reliable enough for my usecase (an option if we follow this is to keep retrying if model generates invalid json) . There was another suggestion to use another assistant with response schema to transform the response. That makes more sense to me. Where the response is guranted to be in a structure.

7

To get back at the heart of your concern: if you are not giving someone a chat experience, but are only making one knowledge search to receive one JSON, if you did use functions to get a JSON response, you could just abandon the thread after getting your function call that looks like a final output.

Assistants locks you into providing a tool return if you want to continue a chat with a thread instead of sending cancel to your run. You can guide that (unnecessary additional) turn, by returning instructions of what to do, such as “response was displayed to user. Your only output now is ‘How may I be of further assistance?’”