Fine-Tunings with function calls

API
1

Hello folks,

I’m struggling with fine-tunings function calls, attempting to train the base model gpt-3.5-turbo-1106 in order getting a function call as response sending a specific commands/phrases. In the detail, i’m trying to instruct the model to send a function “makePhoneCall” when i send the user message content ( Italian language ) “Telefona a ” and force it to reply with function call only if it receives this user message content and exclude a similar semantic phrase as “Chiama a ”. My dataset contains a lot of examples with different completations as following:

{“messages”:[{“role”:“user”,“content”:“telefona a Giovanna per la riunione”},{“role”:“assistant”,“content”:null,“function_call”:{“name”:“makePhoneCall”,“arguments”:“{"contact": "Giovanna"}”}}]}
{“messages”:[{“role”:“user”,“content”:“telefonami Mattia per l’imbarco”},{“role”:“assistant”,“content”:null,“function_call”:{“name”:“makePhoneCall”,“arguments”:“{"contact": "Mattia"}”}}]}
{“messages”:[{“role”:“user”,“content”:“telefona Nico per la palestra”},{“role”:“assistant”,“content”:null,“function_call”:{“name”:“makePhoneCall”,“arguments”:“{"contact": "Nico"}”}}]}

And other exact examples to decline similar commands/completations:

{“messages”:[{“role”:“user”,“content”:“Chiama a Giovanna per la riunione.”},{“role”:“assistant”,“content”:“Non posso chiamare Giovanna, ma se vuoi al limite posso solo mandargli un messaggio o fare una telefonata ?####”}]}
{“messages”:[{“role”:“user”,“content”:“Chiamami Mattia per l’imbarco.”},{“role”:“assistant”,“content”:“Non posso chiamare Mattia, al limite posso solo mandarle un messaggio o fare una telefonata ?####”}]}
{“messages”:[{“role”:“user”,“content”:“Chiama Nico per la palestra”},{“role”:“assistant”,“content”:“Non posso chiamare Nico, al limite posso solo mandare un messaggio o fare una telefonata ?”}]}

I’ve a lot of this kind of examples in my dataset. The problem is that the trained model i got does not distinguish these two different commands/phrases and it always replies with “makePhoneCall” function in both scenarios.

I’m experimented increasing epochs number training in fine-tunings, even to 20-30, but nothing changed.

Could give me some tips it about ? I’m new one to these approachs about fine-tunings.

Thank you guys.

2

I don’t know the full schema but you likely need to prompt the function_call better. The function description matters quite a bit (along with all the parameters)

1 Like
3

Yes. But the problem i investigated is related to the semantic nature of terms in Italian language ( “Chiama” == “Telefona” ) are pretty similar; the model creates a strong association to makePhoneCall examples with “Telefona” and it ignores the examples about “Chiama” even if i create a thousands of examples and contexts in order to train the model to reject the makePhoneCall it about, when “Chiama X” is detected. It seems there’s no way to “break” this inference, neither training with 20-30 epochs leading to overfitting. Of course, it will be a solution but it’s ignored by me at moment.