ChatGPT ALWAYS call function

API
,
1

I’m playing with functions and finding that in some cases, ChatGPT ALWAYS calls a function, if one is provided.

E.g. if the message is:
“”"
The current weather in Boston, MA is 55F with sunny skies.
What is the current weather in Boston?
“”"
and you provide the function (from the example):
get_current_weather …

It will respond with a function call to get the current weather.
I made this very obvious example for demonstration purposes. A “real” use case would be that ChatGPT is asked to answer a customer’s question, based on on order details that are already provided in the prompt, but it still decided to call the get_order_details function.

… adding example (I use Java):

OpenAiChatRequest request = new OpenAiChatRequest();
		request.setSystemMessage(
				"Try to write a response to the message by using the provided information. "
				+ "Only call a function if the required data is not provided. "
				+ "Don't make assumptions about what values to plug into functions.");
		request.addUserMessage(
				"Customer Question: When will my order (#12345) arrive? I ordered it several days ago. -Tom"
				+ "Order Details: { \"order_id\":12345, \"arrival_date\":\"10/20/23\"}, today is 10/19/23");

		// Using FunctionBuilder to create a Function
		OaiFunctionDescription function = new FunctionBuilder()
				.withName("get_order_details")
				.withDescription("Get the order details for a given order number")
				.withAvailableArg("order_id", "The ID of the order", true).buildForOai();

		request.addFunction(function);

		OpenAiChatResponse response = OaiApi.makeChatRequest(request, 30000);
		System.out.println(response);

Response:

{
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_order_details",
          "arguments": "{\n  \"order_id\": \"12345\"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],
 ...
}
2

This happens, for example, when your function call returns nothing or an ambiguous result, or you have forgotten to add the function result to the conversation.

1 Like
3

@co_ting It will be helpful if you can share a snippet to reproduce the issue, and we can take a deeper look.

4

Hi Frank, this happens on the first message. Before any function call is made.

5

Hi kjordan, sure I added my code above.

6

Since AI can’t really perform calculations, it might not be a good idea to rely on it to accurately determine when one date comes before another. And perhaps it ‘consciously’ avoids doing so, which is why it always calls the function.

7

Summarized:

function name: “get_order_details”
user: when will my order arrive?

topic’s question: why does it call the function?

:woman_shrugging:

8

@co_ting I tried and did some tests based on your code. Managed to let the model decide whether to call the function.

Two changes are made:

  1. Tweak the system prompt with more specific requirement, tell the model what’s in the order detail data.
  2. Separate the context information with another user message instead merged in the same message.

Here’s the one with context data in a separate message.

2023-10-25 at 2.39 PM
2023-10-25 at 2.39 PM3352×1302 336 KB

Here’s the function call case.

2023-10-25 at 2.39 PM
2023-10-25 at 2.39 PM2896×1130 267 KB

9

@kjordan , wow - you really spent some time on this - thanks!
I can reproduce the results with the shorter inputs, but not when inserting all the information in different messages. I did notice that this seems to be a gpt 3.5 problem, and it’s getting confused when presented with too much data. gpt 4 does not have this problem.

10

Had the same issue. To fix it I created a separate request that send function result to chatgpt API, but doesn’t contains functions array. But anyway it looks strange that chatgpt asks to run function in response to message with this function result.