In the json response for a function call, Will the content always be NONE?
Or will it be possible that I got a Content Answer and a function call at the same time?
Is it guaranteed that it’s either Content or a Tool_Call, not both at the same time?
No answer to this one, anybody? !
Maybe My question is not clear, here it is again:
When I submit a question with functions to OpenAI, I could get a function call, or simply a text answer.
My question is: Is it Possible for me to get both? i.e., in the response from GPT, there is content and a tools_call?
I am asking because occasionally I am getting both. It seems contradicts to the document, but I am not sure. It’s quite important for me to structure my code.
When you send a message to OpenAI, let’s say ChatCompletion. It will return you only the parameters as JSON output and not any messages.
No. It’s either/or.
It will either respond to give you completion content or a function call but not both at the same time.
It can however give you multiple parallel function calls in the same response though!
You answer your own question.
The AI and the backend mechanism has the ability for the AI to start by writing a “content” to the user, and then separately to invoke a tool token and address a tool message to a recipient in the same response.
The AI is trained to do this less, but I can immediately train it to do it more just for this reply.
1 Like
I stand corrected.
How useful is this though?
Isn’t it better for the function calling to resolve and the bot to “shut up” until it has the information to respond with?
Having coded a chatbot I prefer the predictability of the out of the box (either/or) behaviour.
You have to track the run requires_action status, use the tool call, submit tool output, after output is submitted successful, retrieve assistant message.
You might like an AI that talks about how it is going to address your problem by running scripts or accessing services for a minute instead of just seeing nothing or not knowing what it is up to…with a UI that can connect to http_connection.close().
Reasoning about what it is going to do can help it do the job better.
1 Like
I could achieve that mechanically without the LLM though - for example I could post “using the calculator” verbatim from my local function code without needing the LLM to provide those words.
Brainstorming buddy
Emitting language to the user before invoking a tool can be beneficial in several ways:
-
Setting Context and Expectations: By providing initial information or explanations, the AI can prepare the user for the forthcoming action. This helps in managing the user’s expectations about what the tool will accomplish or the kind of information it will retrieve.
-
Refining Understanding: Articulating thoughts before using a tool allows the AI to process and clarify the user’s request internally. This self-explanation can lead to a more accurate and relevant tool invocation, as it helps the AI to identify the precise requirements.
-
Providing Preliminary Information: The AI might have some immediate insights or general information based on its training data. Sharing this before using the tool can offer the user immediate value, which can be supplemented or confirmed after the tool’s output is integrated.
-
Enhancing User Experience: Initial communication can make the interaction feel more natural and engaging. It reassures the user that their request is being addressed and can reduce any uncertainty during the pause while the tool is being used.
-
Error Handling and Confirmation: By summarizing the intended action, the AI can implicitly check for any misinterpretations of the user’s request. While the user can’t interact at that moment, this step can help prevent errors by ensuring the AI’s understanding aligns with typical user expectations.
-
Complex Task Decomposition: For multifaceted requests, the AI can outline the steps it will take, which may involve multiple tool usages. This not only organizes the AI’s approach but also keeps the user informed about how their request is being handled.
By incorporating these preliminary steps, the AI can improve the effectiveness of the tool usage and enhance the quality of the final answer, leading to a more satisfactory user experience.
The most important thing, to continue this topic’s focus, is to write your code to handle this case of both being received.
You can’t just use the first token out of a stream as a function/no-function switch - you need to collect the whole response and reassemble tool or function chunks (why not code for both). Likewise for a non-streaming parser, you can’t short circuit and say ‘if tool response, else user response’.
Then you’re ready to ask the AI for pre-response, or to deny it, in any case.
I think this a style thing.
I’ve not found my bot particularly wanting with my use cases in terms of reasoning for choice of function.
The functions rarely take that long but noted it might be a good idea to post a “I’m doing this” for long running functions. That doesn’t have to rely on the LLM though.
I will stick with the strict choice of function or content for the time being. It keeps my code clean .
I agree with you. I think this feature (or I rather call it a bug) sucks.
Here is the problem: There’s no way my code can handle this in stream mode.
When we use stream mode, we stream the messages to the end user before I know it’s a function call, it’s too late for me to recall those messages from the user.
This is really unfortunate, I thought only Claude does this, as OPENAI doesn’t do this often, until it does.
If it really does WANT to give me a reason why it’s calling a function, it should include that message inside the tool_call response, not the content. Thus I will catch the tool call first, then got the reason why!