I’ve been trying to rework my code to adhere to the new standards for tools over the old function mechanics, so far it’s ok but there are three things that end being limiting for me:
The fact that these messages need to be immediately after each other is a bit limiting, it means I need to build a lot of code to ensure system or assistant messages are delivered in a very specific order. For example: A function call is made, and part of the data from that function call is sent directly to the user prior to executing the function itself, but now I cannot follow the logical order, I have to insert this message before the function call, where the AI loses a bit of coherency.
The fact that there cannot be a lone function response: I’m grabbing as much chat history as I can fit into any given conversation, here it would be really useful if the function response was able to stay in context even though the call itself is too many tokens to fit within my limit. Both for coherency of the conversation but also here again I need to build code that parses the messages from my db to look for a tool_call_id and make sure that I remove any follow up where the call doesn’t fit.
The fact that there cannot be a lone function call: Some of the functions I have are essentially asynchronous, one call triggers calls to other functions, so it would be useful to be able to ask the AI to make a tool call with just the original call in context. I’ve had to create segmented portions of interaction that isn’t part of the normal chat history. Edit; Though I should mention this pattern as a solution is working much better for me in practice than the previous 2 points.
Over-all, I’m just slightly confused about these requirements, and I wonder if the benefits of a more strict structure really outweighs the lack of freedom within the requirements.