Function calling uses previous call's values

I have precisely mentioned in the instructions and also in the function descriptions to not use previous arguments, but the bot still uses previous arguments when a function call is made. To an extent that its not even asking me the arguments, just directly using the previous arguments.

@_j @ramn7 Can you guys help? TIA

Hi @suganthan , this is interesting- I don’t think I’ve ever noticed such an issue.

Can you please provide some more details?
I’m guessing this is an assistant, and the issue happens within the same thread (?).
Is it consistent across different cases? Could you please provide a concrete example and details that could potentially be reproduced?

Hi @ramn7 This is not an assistant, I’m using chat completion with function calling. I’m using a flask API for this. I’m having a set of function definitions and at the end of each function call, I’m clearing the conversation history, including the instructions. And I’m sending instructions again, along with the user input in the next flask API call.

Thanks @suganthan for the additional info.

This is interesting, can you please provide a concrete example and flow that could possibly be reproduced?
Not sure I understood what “clearing the conversation history” means, is this not a regular api call?

BTW saw this yesterday Function calling using memory information instead of calling the function, it’s for a different usecase but I guess could potentiall be related.

This is a toned-down generic code that I’m using:

import openai
from dotenv import load_dotenv
import requests
import os
import json
import logging
import random
from flask import Flask, request, jsonify
from flask_cors import CORS, cross_origin

app = Flask(__name__)
cors = CORS(app, origins=["*"])

load_dotenv()

client = openai.OpenAI()
protocol = os.getenv("PROTOCOL")
host = os.getenv("HOST")
port = os.getenv("PORT")

current_dir = os.getcwd()
instructions_loc = os.path.join(current_dir, "instructions.txt")

if instructions_loc is None:
    raise ValueError("instructions not present")

with open(instructions_loc, 'r') as file:
    instructions = file.read()

conversation_history = []

tools = [
    {
        "name": "create_subscriber",
        "type": "function",
        "function": {
            "name": "create_subscriber",
            "description": "This function must create a subscriber for activation, with all fields being mandatory and directly provided by the user. Never call the function with empty arguments. Demand the user provides account number, phonenum, and ims. Do not add any values yourself—use only the user's input. Do not reuse values from previous function calls; always start fresh. If the user provides only one or two fields, immediately demand the remaining fields before calling the function. If the user refers to the required fields by numbering (e.g., 1, 2, and 3), accurately map the values to the correct fields. Allow identical values for all three fields if the user provides them. Strictly follow these instructions: do not pre-fill or assume any parameters.",
            "parameters": {
                "type": "object",
                "properties": {
                    "accountNo": {
                        "type": "string",
                        "description": "Account number of the subscriber"
                    },
                    "phonenum": {
                        "type": "string",
                        "description": "phonenum of the subscriber. Mobile Station International Subscriber Directory Number. This is the identification of a subscriber"
                    },
                    "ims": {
                        "type": "string",
                        "description": "ims stands for International Mobile Subscriber Identity, which is a unique number that identifies a mobile phone subscriber on any cellular network."
                    },
                },
                "required": ["accountNo", "phonenum", "ims"]
            }
        }
    }
]

def create_subscriber(accountNo, phonenum, ims):
    url = (
        f"{protocol}://{host}:{port}/createSubscriber"
    )
    try:
        req_body = {
            "accountNo": accountNo,
            "billUnit": [
                {
                    "bdom": 1
                }
            ],
            "services": [
                {
                    "phonenum": phonenum,
                    "ims": ims
                }
            ]
        }
        logging.error(req_body)
        response = requests.post(url, json=req_body)
        status_resp = json.dumps(response.json(), indent=4)
        resp_json = json.loads(status_resp)
        data = resp_json
        logging.error(response.json())
        if response.status_code == 200:
            offer_details = f"Subscriber created successfully!"
            return offer_details
        else:
            err_msg = data["errorDescription"]
            prompt = f"Context: there was an error and this is the error message: {err_msg}\n\nFormat the error response and provide it back to the user" 
            response = client.chat.completions.create(
                model=os.getenv("MODEL"),
                messages=[{"role": "user", "content": prompt}],
                temperature=float(os.getenv("TEMPERATURE")),
            )
        response_message = response.choices[0].message.content
        conversation_history.append({"role": "assistant", "content": "function calling done"})
        return response_message

    except requests.exceptions.RequestException as e:
        logging.error(e)
        return "Apologies. Unable to process request. Please try again."

@app.route('/chat', methods=['POST', 'OPTIONS'])
@cross_origin()
def chat():
    if conversation_history:
        if conversation_history[-1]["content"] == "function calling done":
            print("removing previous history")
            conversation_history.clear()
            conversation_history.append({"role": "system", "content": instructions})
    input_data = request.json
    user_input = input_data.get('text')
    if user_input != None:
        conversation_history.append(
            {"role": "user", "content": user_input})

    print("conversation_history", conversation_history)
    response = client.chat.completions.create(
        model=os.getenv("MODEL"),
        messages=conversation_history,
        temperature=float(os.getenv("TEMPERATURE")),
        tools=tools,
        tool_choice="auto"
    )

    if response.choices[0].message.content != None:
        conversation_history.append(
            {"role": "assistant", "content": response.choices[0].message.content})

    response_message = response.choices[0].message

    if dict(response_message).get('tool_calls'):

        function_called = response_message.tool_calls[0].function.name

        match function_called:
            case "create_subscriber":
                arguments = json.loads(
                    response_message.tool_calls[0].function.arguments)
                accountNo = arguments.get("accountNo", "")
                phonenum = arguments.get("phonenum", "")
                ims = arguments.get("ims", "")
                response_message = create_subscriber(
                    accountNo, phonenum, ims)

                if response_message != []:
                    return jsonify({"text": response_message})
                elif response_message == []:
                    return jsonify({"text": "Sorry! I couldn't able to create a subscriber. Please try again."})
    else:
        response_message = response_message.content
        return jsonify({"text": response_message})


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

These are the content in instructions.txt:

You can do data related tasks. You can lookup databases and provide information. 
You can create data in database if requested by the user. Based on user's request you should call respective functions.
It is mandatory to not use parameters of your own for function calling. Always ask for all the fields from the user only. Never ever use fields from history or make up parameters.
Using fields from history is only allowed when it is from the current function call instance, not from the previous instance of calling the same function.
Stricly follow the user's requests. Never ever make up parameters or use the parameters from previous calls.
You can wait for user's input for arguments until all the parameters are received, but never use previous request's parameters. 
If the user greets you give them responses. if the user asks something and you don't know or understand that, convey it to the user.
Never return nothing. But also don't answer anything that is outside of this scope. just say you are not capable of it.
Don’t make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.

In this you can see I’m removing the conversation history when a function calling was done. But still the function calling uses previous values. @ramn7

Thanks, I understand now what you meant.

Do you have an example for an input that fails?

When I say create subscriber, it asks for all the parameters and once I give it, it calls the api and everything is perfect. But after I get “subscriber created successfully”, if I say create subscriber again, it uses the old request parameters by itself and calls the function and the api within

OK I couldn’t reproduce with pure API calls.
Then I briefly looked at the code- are you sure the history is cleared correctly before the next call? you call return offer_details before appending “function calling done”.
I didn’t look too carefully so I might be missing something, try to make sure to print the history on the second call to see how it looks.

BTW- model is 4o? what temperature?

1 Like

Thanks @ramn7 actually that was a code issue as you mentioned, the history was not being cleared always. The model that I’m using is gpt-3.5-turbo and temperature is 0.5. I face another issue actually. I have one more function which takes a single parameter. But the bot does not directly detects it. Let’s say the function is to get balance. If I say “get balance”, it asks for the mobile number. If I provide the number alone, it asks me again. and if I provide it again it works. But if I hust say “get balance for 90310” or “get balance” and then “mobile number is 90310”, it is working fine

Glad that helped. Maybe best to mark that as the solution so people seeing this can quickly understand it’s not an issue with the model.

Regarding your second question, I’d try iterating on the function and parameters description, along with the system prompt.
I’d suggest keeping function description purely on what the function does, and leave meta instructions to the system message (such as ‘Demand the user provides…’ and similar from your example).

Sure, will do it. Thanks for the reply!

1 Like