Assistant + function calling + file search

I have created an assistant that has an associated function, plus file search with a vector store. I want the assistant to ALWAYS use my function for final output, as it enforces a strict schema (using the latest schema options).

It SEEMS like what I should be able to do is set “tool_choice” to my custom function. However when I do this it does not use the file_search tool. I have also tried setting it to “required” - but then it actually tries to call a function called “msearch” which isn’t defined - and doesn’t actually use it’s own file_search or my function.

Based on searching the forums and my current understanding, the BEST solution at the moment is to set tool_choice to “auto” and then append an additional instruction into the run that says: “VERY IMPORTANT: ALWAYS call the function with your response.” This seems to do exactly what I want - it will do a file search, use the data in the files to come up with an answer, then format it using the schema in my custom function and make the tool call in the response.

Is my understanding correct? Is this currently the best practice? If so, it’s pretty fragile and undermines the purpose of the new schema functionality, as during my testing it is still possible for it to ignore those very explicit instructions and fail to make the custom tool call.

Any ideas on how to do this better?

Here’s my function definition, btw:

{
  "name": "get_feedback",
  "description": "Feedback",
  "strict": true,
  "parameters": {
    "type": "object",
    "properties": {
      "response_type": {
        "type": "string",
        "description": "response type",
        "enum": [
          "text",
          "numeric",
          "array"
        ]
      },
      "feedback": {
        "anyOf": [
          {
            "type": "string",
            "description": "text feedback"
          },
          {
            "type": "integer",
            "description": "oneof single numeric choice"
          },
          {
            "type": "array",
            "description": "array of numeric choices",
            "items": {
              "type": "integer"
            }
          }
        ]
      },
      "rationale": {
        "anyOf": [
          {
            "type": "null",
            "description": "no rationale needed for text answers"
          },
          {
            "type": "string",
            "description": "for single or array choices, explain how you came to your answer."
          }
        ]
      }
    },
    "additionalProperties": false,
    "required": [
      "feedback",
      "response_type",
      "rationale"
    ]
  }
}
3 Likes

Hey,

thank you very much for sharing your findings so far.

Have you already found a better solution? I’m at exactly the same point.

I also noticed the following: a few days ago there was an update regarding the output of the searched / used file search chunks. Unfortunately, I am not able to display this via the run step chunks in my stream. If I manually send a curl request to a thread (see https://platform.openai.com/docs/assistants/tools/file-search “Inspecting file search chunks”), I unfortunately don’t see anything interesting in the payload. Only in the Playground can I see in the logs which file search chunks have been used.

Thank you guys for any more info or details. Cheers

PS: It would be a nice feature for assistants api to provide the opportunity to pass an array with specific required tool functions instead of only one.