Use function only in last message

API
, ,
1

We are writing an internal chat tool at our company and I would like to thank everyone here because I have referred to this forum so many times and it has saved me SO much time.

Anyway, we are starting to implement function calling. As the chat proceeds, it builds an array of the conversation and appends the last prompt at the end like many custom chat apps do where remembering the history of the conversation is vital.

I was tinkering using the weather example in the docs and noticed that it is inconsistent in which message to pass to the function, sometimes it sends them all. Is there a way to tell it to only apply functions to the last message or is there a way to have it ignore specific messages?

Here is the array of messages I am sending and in their order (assistant is previous responses from OpenAI):

image
image894×357 40.5 KB

Of course this muddies the response.

It would be great if we could do:

{"role":"user","content":"hello there",ignoreFunctions:true}

Or something like this when sending up the prompts.

2

The AI that gets the result of the function-calling should have in conversation history

-again the inclusion of all functions

  • the system message
  • all prior chat
  • the user’s question again
  • a assistant simulation of the function call, which appears like #weather({"query": "miami})
  • a function role message, with the return from the API

By seeing each iteration of function call and results, it can then decide when it has gathered enough infomation to answer, without being caught in a loop. It is given the functions again to understand how they operated.

It is possible to have a different system prompt in the second call, one which discourages more function calls unless necessary.

2 Likes
4

I’ve found that it’s important to at least have the result of the last function call in the conversation. However, the specifics really depend on the combination of function calls within the conversation. This became apparent to me when working with my code interpreter. If a call failed and the model tried to regenerate the code, it would get stuck in an endless loop, especially if the function result and error message remained in the conversation. So, it’s hard to make a blanket statement about when to include the result and when not to. It might be best practice to avoid lengthy conversations with numerous function calls. Alternatively, one must put in a lot of effort to organize the flow of the conversation.
I’ve implemented it in such a way that for every function called in the conversation, I only return the latest result to the model. This approach might be how a human would handle it as well.

{
    "role": "system",
    "content": "You are a helpful assistant."
}
{
    "role": "user",
    "content": "What is the weather in Berlin?"
}
{
    "role": "assistant",
    "content": "The current weather in Berlin is 22 degrees Celsius. The forecast indicates it's sunny and windy."
}
{
    "role": "user",
    "content": "What is the weather in Cologne?"
}
{
    "role": "assistant",
    "content": "The current weather in Cologne is 22 degrees Celsius. The forecast indicates it's sunny and windy."
}
{
    "role": "user",
    "content": "What is the weather in Frankfurt?"
}
{
    "role": "assistant",
    "content": "The current weather in Frankfurt is 22 degrees Celsius. The forecast indicates it's sunny and windy."
}
{
    "role": "user",
    "content": "What is the weather in New York?"
}
{
    "role": "function",
    "name": "get_current_weather",
    "content": "{\"location\": \"New York\", \"temperature\": \"22\", \"unit\": \"celsius\", \"forecast\": [\"sunny\", \"windy\"]}"
}
{
    "role": "assistant",
    "content": "The current weather in New York is 22 degrees Celsius. The forecast indicates it's sunny and windy."
}
1 Like
5

Yes, it is reasonable:

  • while the AI is still calling functions to attempt to obtain a solution that satisfies input, full consecutive history of function calling results is required (along with inputs that generated them). What has failed and what knowledge has been gained should all be seen.

  • once the AI has been able to answer the user, the function usage does not need to be retained.
    – it is unlikely to teach the AI how to use functions better as a possible reason to keep,
    – instead past returns may be seen as still informing the AI’s latest answer

1 Like
6

One could consider creating a context wrapper: a dedicated conversation with its own system prompt focused solely on managing the conversation.

Edit:
Ah, okay, with that sentence, you probably already suggested it that way.

It is possible to have a different system prompt in the second call, one which discourages more function calls unless necessary.

7

Thanks. The " a function role message, with the return from the API" really helped. I did not find that in the documentation. Thanks again!

1 Like
8

This step is not particularly understood, how exactly can I simulate this function call. does it mean spelling out a call-like string after the assistant’s answer?

9

At the time of writing, there was no mechanism documented of how one would tell the AI in conversation history what it had produced before to call a function.

Now there is an API method which I explain - still hiding the actual language the AI uses to call a function.

You will see in that post that you are sending the same chat history and user question as before, but also adding the function call the AI used and the return value from the function to the end of your new messages input.

2 Likes
10

Sincere thanks for such a speedy and valuable response.