This is feedback from a extremely frustrated developer. The model just initiates tools when it likes to. Even when you heavily instruct it in both system message and the tool, the tool is still initiated. I have a really hard time controlling that, and it is so important for the user experience, that the model don’t suddenly go off and start initiating stuff the changes the UI.
Anyone else having problems getting the model to follow the instructions when it comes to usage of tools?
Would love some help!
Totally agree it can be frustrating sometimes…
Without knowing your exact use-case, here is some options I found helpful:
- I found system messages similar to the one used in this example to be somewhat useful.
Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.
- I found that sometimes instructional, structured prompts work better than function calling (basically asking in the system message to respond in a json with the arguments of the function call as attributes, when relevant).
** probably depends on the use-case.
- Building a “confirmation” mechanism into the end-user flow- makes sure unwanted tools are not actually called, and also enables your learning from it in order to improve the prompt.
Hope that’s somewhat relevant and helpful.