Fine-tuning with Responses API, file search and function calling

API
, , ,
1

Is there a way to fine-tune a model for use with the Responses API (specifically, for an application that uses function calling and file search)? I was hoping to take real user queries and fine-tune it to better call the correct function tool…and also take function tool responses and fine-tune the resulting message back to the user.

The fine-tuning guides all reference the Chat Completions API, and the required jsonl format is for chat completions. I could try to convert all my Responses API history into the Chat Completions format and build a jsonl file with that, but I’m skeptical as to whether the resulting fine-tuned model would then work when I try to use it in the Responses API setting. Plus, chat completions doesn’t support file search, so I guess I’d have to avoid any of the queries where file search was an appropriate response.

If fine-tuning and responses API are just incompatible right now, any idea if it will become an option in future?

And yes, I’ve done a lot of prompt engineering already and will continue to do so!

2 Likes
2

Have you tried it yourself? I’m also feeling a bit skeptical at the moment, but I haven’t tested it with the Responses API yet. The Assistant API only supported GPT-3.5-turbo for fine-tuning with file search, so I’m curious how responses will performs.

3

You can produce fine tuning examples on functions, which is one type of tool that you can add specifications to. The model is employed similarly between Responses and Chat Completions.

You cannot replicate placement of tool specifications needed for internal Assistants or Responses tools, nor patterns of internal tool calling.

Thus, you can use fine-tuning to somewhat understand a function call better, to differentiate between your “product_search” and “knowledge_base”. However, you won’t be able to fill the training examples with real-looking calls or returns for an internal tool such as file search.

Fine tuning with functions is a balancing act that can only be done with experimentation – will you break function calling even worse?

Fine-tuning can damage the model quality for following instructions such as the internal tool specification, but inference is not blocked on Assistants, and file search is still an option on a newer AI model.

Use a recent fine-tuning, with file search:

A configuration interface shows settings for "ChatAPI with file_search," including system instructions, a selected model (ft:gpt-4o-2024-08-06), and an enabled File Search tool. (Captioned by AI)
A configuration interface shows settings for "ChatAPI with file_search," including system instructions, a selected model (ft:gpt-4o-2024-08-06), and an enabled File Search tool. (Captioned by AI)365×429 13 KB

Receive usage of the tool:

The image shows a log output of a JSON response detailing the results of a completed "file_search" tool call, including metadata such as completion status, timestamps, file name, score, and attributes. (Captioned by AI)
The image shows a log output of a JSON response detailing the results of a completed "file_search" tool call, including metadata such as completion status, timestamps, file name, score, and attributes. (Captioned by AI)471×581 7.81 KB

The playground even has background feature data:

    {
      "object": "model",
      "id": "ft:gpt-4o-2024-08-06:xxx:yyy:zzz",
      "api_version": null,
      "streaming_enabled": true,
      "image_inputs_enabled": true,
      "reasoning_effort_enabled": false,
      "retrieval_enabled": true,
      "code_interpreter_enabled": true,
      "browser_enabled": false,
      "sampling_params_enabled": true
    }
4

You can only fine-tune on functions you bring yourself, so some ideas are:

  • Implement the tools yourself and fine-tune with those.
  • Or, introduce diverse examples of functions to train the model to be flexible and respond better to OpenAI’s tools.

Personally, I’d just save some money and implement the tools on my own.