The single example is kind of nonsense for fine-tune on function calling. I have a feeling just as much attention was paid in implementing the fine-tune endpoint for training on functions as in writing the example: nearly none.
“Get similarly formatted responses even when the full function definition isn’t present” Oh, really, no need to include a function to get an endpoint that will emit them? BS.
This is me including a dummy function “disabled” with my fine-tune model, and then injecting the AI text specification of a google_for_answers function right into the system prompt:
{
“message”: {
“role”: “assistant”,
“content”: null,
“function_call”: {
“name”: “google_for_answers”,
“arguments”: “{\n "query": "2023 Oscar winners"\n}”
}
},
“finish_reason”: “function_call”
}
But what if I simply don’t have a dummy function call? My fine-tune model now no longer has the knowledge of what a function is:
“message”: {
“role”: “assistant”,
“content”: “I’m sorry, but as an AI language model, I don’t have real-time information or the ability to browse the internet. My training only goes up until September 2021, so I don’t have access to the list of Oscar winners for 2023. I recommend checking reliable news sources or the official Oscars website for the most up-to-date information on the 2023 Oscar winners.”
},
“finish_reason”: “stop”
Have they trained two different models based on my fine tune, one that won’t obey or understand the function specification?
So I can fine-tune on function-calling and pay 8x for what? To reduce the description by a few tokens?
Then there is the single example given in the quickstart:
Format your examples as shown, with each line including a list of “messages” and an optional list of “functions”:
So the list of functions is optional when fine-tuning for function-calling? Then you are guaranteed to have a different system prompt when you try to use that model (because the function specification is in fact mandatory when using), thus having a lower quality of “identity” to matching your model with your tune.
Then of course they disavow every sales point made: “If your goal is to maximize the correctness of the function calling output, we recommend using the same function definitions for both training and querying the fine-tuned model.”
Then they have you train the AI like this on how the response “should be seen” in chat history, using a special format:
{"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celcius\"}"}}
The only thing: they continue to obfuscate the calling language and tokens, so YOU can’t actually put this into chat history the same way as the special notation, further degrading the fine-tune following of functions. Do fine-tune that way, and your conversation history cannot replicate it to activate your AI response to a return. Do it your own way, and ignore their fake function call json, and you get an AI that you can now train on not being able to call functions the right way because you filled its fine-tune with the wrong calling method.
–
Just piss-poor. Give us the actual AI language in and out of functions. Give us the special tokens and we choose when to block them or insert them. Give us endpoints where we can turn on the function return with a boolean, or instead see what the AI is producing. Let me put every single token into my ChatML container and fill the raw context (instead of you screwing up 0301 with new garbageML) Give us models that you don’t continue to screw with daily.