Openai tools does not work as expected

I am currently exploring the capabilities of OpenAI tools to determine if they can autonomously identify and arrange the necessary functions in the correct order to address a user’s request. My test involves a code with four distinct functions. However, I’m facing a challenge where the final function in the sequence does not seem to receive the complete list of friends as its parameter.

I’m curious if anyone might have insights into this issue. Additionally, I’m uncertain about the proper way to conclude the sequence of tool usage. For this, I’ve been utilizing the completion API to signal the end of a function call. I am thinking of the use case where the tools can be 100’s of function and if function calling should be able to handle or not. Appreciate if anyone has any insights

https://gitlab.com/-/snippets/3641029/raw/main/openai_function_calling.py

import os
import openai
from openai import OpenAI
import pdb

from datetime import date
import random
import traceback

openai.api_key = os.getenv('OPENAI_API_KEY')

system_prompt = """You are a function router AI. You will build a knowledge graph of functions and their inputs and outputs. You will parse input query and 
find the correct sequence of functions to call to return the final correct result. Since the functions may be executed in a sequential manner, you do not need to ask for confirmation on proceeding. Assume user intends to execute all the functions needed to answer the query. """

tools = [
    {
        "type": "function",
        "function": {
            "name": "find_todays_date",
            "description": "Find today's date",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": []
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "find_list_of_friends",
            "description": "Find a list of friends",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": []
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "find_birthdays_of_friends",
            "description": "Find birthdays of friends given a list of friends",
            "parameters": {
                "type": "object",
                "properties": {
                    "friends_list": {
                        "type": "array",
                        "items": {
                            "type": "string"
                        },
                        "description": "A list of friends"
                    },
                },
                "required": ["friends_list"]
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "send_birthday_greetings",
            "description": "Send email greetings if today is a friend's birthday",
            "parameters": {
                "type": "object",
                "properties": {
                    "birthdays_list": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "friend": {"type": "string", "description": "Name of the friend"},
                                "date_of_birth": {"type": "string", "description": "Date of birth of the friend"}
                            },
                            "description": "A list of friends' birthdays"
                        }
                    },
                    "today": {
                        "type": "string",
                        "description": "The current date in a specific format, e.g., '2024-01-21'"
                    }
                },
                "required": ["birthdays_list", "today"]
            }
        }
    }

]

# Dummy function to simulate finding today's date
def find_todays_date():
    # Returns today's date
    return str(date.today())

# Dummy function to simulate finding a list of friends
def find_list_of_friends():
    # Fake data: returns a list of mock friend names
    return ["Alice", "Bob", "Charlie", "Diana", "Eve"]

# Dummy function to simulate finding birthdays of friends
def find_birthdays_of_friends(friends_list):
    # Fake data: assigns a random birthday to each friend
    return {friend: f"199{random.randint(0, 9)}-0{random.randint(1, 9)}-{random.randint(10, 28)}" for friend in friends_list}

# Dummy function to simulate sending birthday greetings
def send_birthday_greetings(birthdays_list, today):
    # Fake operation: returns a success message for each friend
    for birthday in birthdays_list:
        friend = birthday["friend"]
        date_of_birth = birthday["date_of_birth"]
        print(f"Sent birthday greeting to {friend} born on {date_of_birth}")
    return "Greetings sent"



def get_response(messages):
    client = OpenAI()
    response = client.chat.completions.create(
            model="gpt-3.5-turbo-1106",
            messages=messages,
            tools=tools,
            tool_choice="auto",  # auto is default, but we'll be explicit
        )
    return response


def run_tools(tool_calls, messages):
    fnc_results = []
    for t in tool_calls:
        fnc = t.function.name
        args = t.function.arguments
        args = eval(args)
        print(f"Executing {fnc}..")
        fnc_result = eval(f"{fnc}(**args)")
        msg = f"Function {fnc} with arguments {args} returned {fnc_result}."
        print(msg)
        messages.append({"role": "tool", "tool_call_id": t.id, "name": fnc, "content": str(fnc_result)})
    return messages


def check_is_complete(query, response_content):
    try:
        # check if done
        if response_content == None:
            return False
        client = OpenAI()
        messages = []
        system_prompt = "You check the status of last message. You only check if the message is indicative of more steps needed or not. If complete respond yes. Otherwise, respond no."
        message = f"""Last message received is ####\n {response_content} ###\n. 
                    Typically when complete, message would indicate All steps have been successfully completed
                    Check if the last message is implying more steps are needed or if all the steps are complete. please respond with 'yes' if more steps are needed. Otherwise, please respond with 'no'."""
        messages.append({"role": "system", "content": system_prompt,})
        messages.append({"role": "user", "content": message,})

        response = client.chat.completions.create(
                model="gpt-3.5-turbo-1106",
                messages=messages
            )
        print(f"\n\n####check_is_complete: last_message is {response_content}. response was {response}\n\n")
        is_done = 'yes' in  response.choices[0].message.content.lower()
        print(f"Is done? {is_done} from {response_content}.")
        return is_done
    except Exception as e:
        print(e)
        print(traceback.format_exc())
        pdb.set_trace()
        return False

def check_is_complete(query, response_content):
    try:
        # Verification step
        if response_content is None:
            return False
        client = OpenAI()
        messages = []
        system_prompt = "As an AI assistant, your task is to determine if a message indicates completion of all necessary steps or if additional actions are required. Reply 'yes' if more steps are needed based on the message, or 'no' if it indicates that everything is complete. Only respond yes or no text"
        message = f"""I've received the following message: '{response_content}'. 
                    Generally, a message signaling completion would explicitly state that all required steps have been successfully finished. 
                    Please review whether this message suggests that there 
                    are additional steps to be taken or if it signifies the completion of tasks. 
                    Note that there might be other messages detailing what has been completed 
                    regarding the original query '{query}'. However,
                    my focus is solely on determining the completion status of the steps. 
                    Respond with 'yes' if additional steps are necessary, or 'no' if 
                    everything is complete. Only respond yes or no."""
        messages.append({"role": "system", "content": system_prompt})
        messages.append({"role": "user", "content": message})

        response = client.chat.completions.create(
            model="gpt-3.5-turbo-1106",
            messages=messages
        )
        #print(f"\n\n####check_is_complete: last_message is {response_content}. response was {response}\n\n")
        do_more = 'yes' in response.choices[0].message.content.lower()
        #print(f"Do more steps need to be taken? {do_more} based on '{response_content}'.")
        return not do_more
    except Exception as e:
        print(e)
        return False


query = "Find a list of friends and send birthday greetings if today is a friend's birthday"
chain_message = "Find a list of friends and send birthday greetings if today is a friend's birthday"
messages = []
messages.append({"role": "system", "content": system_prompt,})
messages.append({"role": "user", "content": chain_message,})

while True:
    try:
        response = get_response(messages)
        response_message = response.choices[0].message
        response_content = response_message.content
        tool_calls = response_message.tool_calls
        #print(response_message)
        #print(response_content)
            
        is_complete = check_is_complete(query, response_content)
        if is_complete and tool_calls == None:
            break
        if not tool_calls:
            continue
        assistant_message = {"role": "assistant", 'tool_calls': tool_calls}
        messages.append(assistant_message)

        messages = run_tools(tool_calls, messages)
        next_question = "What step should I take next?"
        messages.append({"role": "user", 'content': next_question})
    except Exception as e:
        print(e)
        print(traceback.format_exc())
        pdb.set_trace()
        break
1 Like
python3.11 openai_function_calling.py 
Executing find_list_of_friends..
Function find_list_of_friends with arguments {} returned ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve'].
Executing find_todays_date..
Function find_todays_date with arguments {} returned 2024-01-21.
Executing find_birthdays_of_friends..
Function find_birthdays_of_friends with arguments {'friends_list': ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve']} returned {'Alice': '1992-05-27', 'Bob': '1996-04-19', 'Charlie': '1999-04-21', 'Diana': '1994-01-21', 'Eve': '1997-02-17'}.
Executing send_birthday_greetings..
Sent birthday greeting to Diana born on 1994-01-21
Function send_birthday_greetings with arguments {'birthdays_list': [{'friend': 'Diana', 'date_of_birth': '1994-01-21'}], 'today': '2024-01-21'} returned Greetings sent.

This is what my run looks like. For some reason, birthdays_list is not fully populated with all the data.

Here’s my observation:

  • find_birthdays_of_friends returns a dictionary with friends’ names as keys and their birthdates as values.
  • However, the send_birthday_greetings function expects a list of dictionaries, each containing friend and date_of_birth keys.
  • It seems like the conversion from the output of find_birthdays_of_friends to the format expected by send_birthday_greetings is not being handled correctly.
2 Likes

That’s a good find. Thanks. I made some changes to the function to be more explicit and also added better description. Still it it doesn’t seem to work correctly. Wonder if anyone has successfully used functions in a more complex scenario over the toy weather program.

import os
import openai
from openai import OpenAI
import pdb

from datetime import date
import random
import traceback


MODEL_NAME = "gpt-4-1106-preview"

# Lower values for temperature result in more consistent outputs (e.g. 0.2), while higher values generate more diverse and creative results (e.g. 1.0)
DETERMINISTIC_TEMPERATURE = 0.2
openai.api_key = os.getenv('OPENAI_API_KEY')

system_prompt = """You are a function router AI. You will build a knowledge graph of functions and their inputs and outputs. You will parse input query and 
find the correct sequence of functions to call to return the final correct result. Since the functions may be executed in a sequential manner, you do not need to ask for confirmation on proceeding. Assume user intends to execute all the functions needed to answer the query. """

tools = [
    {
        "type": "function",
        "function": {
            "name": "find_todays_date",
            "description": "Find today's date",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": []
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "find_list_of_friends",
            "description": "Find a list of friends. Returns an array of strings corresponding to the names of friends",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": []
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "find_birthdays_of_friends",
            "description": "Find birthdays of friends given a list of friends. Returns an array of objects with two keys 'friend' and 'date_of_birth' in each item corresponding to a friend",
            "parameters": {
                "type": "object",
                "properties": {
                    "friends_list": {
                        "type": "array",
                        "items": {
                            "type": "string"
                        },
                        "description": "A list of friends"
                    },
                },
                "required": ["friends_list"]
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "send_birthday_greetings",
            "description": "Send email greetings if today is a friend's birthday. All inputs needed should be passed. First Input parameter is an array of objects with two keys 'friend' and 'date_of_birth' in each item corresponding to a friend and second input parameter is today's date. Returns a string indicating success or failure",
            "parameters": {
                "type": "object",
                "properties": {
                    "birthdays_list": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "friend": {"type": "string", "description": "Name of the friend"},
                                "date_of_birth": {"type": "string", "description": "Date of birth of the friend"}
                            },
                            "description": "Each array item contains an object with two keys 'friend' and 'date_of_birth' in each item corresponding to a friend"
                        }
                    },
                    "today": {
                        "type": "string",
                        "description": "The current date in a specific format, e.g., '2024-01-21'"
                    }
                },
                "required": ["birthdays_list", "today"]
            }
        }
    }

]

# Dummy function to simulate finding today's date
def find_todays_date():
    # Returns today's date
    return str(date.today())

# Dummy function to simulate finding a list of friends
def find_list_of_friends():
    # Fake data: returns a list of mock friend names
    return ["Alice", "Bob", "Charlie", "Diana", "Eve"]

# Dummy function to simulate finding birthdays of friends
def find_birthdays_of_friends(friends_list):
    # Fake data: assigns a random birthday to each friend
    data_arr = []
    for friend in friends_list:
        data = {"friend": friend, "date_of_birth": f"199{random.randint(0, 9)}-0{random.randint(1, 9)}-{random.randint(10, 28)}"}
        data_arr.append(data)
    return data_arr

# Dummy function to simulate sending birthday greetings
def send_birthday_greetings(birthdays_list, today):
    # Fake operation: returns a success message for each friend
    for birthday in birthdays_list:
        friend = birthday["friend"]
        date_of_birth = birthday["date_of_birth"]
        print(f"Sent birthday greeting to {friend} born on {date_of_birth}")
    return "Greetings sent"



def get_response(messages):
    client = OpenAI()
    response = client.chat.completions.create(
            model="gpt-3.5-turbo-1106",
            messages=messages,
            tools=tools,
            tool_choice="auto",  # auto is default, but we'll be explicit
        )
    return response


def run_tools(tool_calls, messages):
    fnc_results = []
    for t in tool_calls:
        try:
            fnc = t.function.name
            args = t.function.arguments
            args = eval(args)
            print(f"Executing {fnc}..")
            fnc_result = eval(f"{fnc}(**args)")
            msg = f"Function {fnc} with arguments {args} returned {fnc_result}."
            print(msg)
            messages.append({"role": "tool", "tool_call_id": t.id, "name": fnc, "content": str(fnc_result)})
        except Exception as e:
            print(e)
            print(traceback.format_exc())
            pdb.set_trace()
    return messages


def check_is_complete(query, response_content):
    try:
        # check if done
        if response_content == None:
            return False
        client = OpenAI()
        messages = []
        system_prompt = "You check the status of last message. You only check if the message is indicative of more steps needed or not. If complete respond yes. Otherwise, respond no."
        message = f"""Last message received is ####\n {response_content} ###\n. 
                    Typically when complete, message would indicate All steps have been successfully completed
                    Check if the last message is implying more steps are needed or if all the steps are complete. please respond with 'yes' if more steps are needed. Otherwise, please respond with 'no'."""
        messages.append({"role": "system", "content": system_prompt,})
        messages.append({"role": "user", "content": message,})

        response = client.chat.completions.create(
                model="gpt-3.5-turbo-1106",
                messages=messages,
                temperature=DETERMINISTIC_TEMPERATURE,
            )
        print(f"\n\n####check_is_complete: last_message is {response_content}. response was {response}\n\n")
        is_done = 'yes' in  response.choices[0].message.content.lower()
        print(f"Is done? {is_done} from {response_content}.")
        return is_done
    except Exception as e:
        print(e)
        print(traceback.format_exc())
        pdb.set_trace()
        return False

def check_is_complete(query, response_content):
    try:
        # Verification step
        if response_content is None:
            return False
        client = OpenAI()
        messages = []
        system_prompt = "As an AI assistant, your task is to determine if a message indicates completion of all necessary steps or if additional actions are required. Reply 'yes' if more steps are needed based on the message, or 'no' if it indicates that everything is complete. Only respond yes or no text"
        message = f"""I've received the following message: '{response_content}'. 
                    Generally, a message signaling completion would explicitly state that all required steps have been successfully finished. 
                    Please review whether this message suggests that there 
                    are additional steps to be taken or if it signifies the completion of tasks. 
                    Note that there might be other messages detailing what has been completed 
                    regarding the original query '{query}'. However,
                    my focus is solely on determining the completion status of the steps. 
                    Respond with 'yes' if additional steps are necessary, or 'no' if 
                    everything is complete. Only respond yes or no."""
        messages.append({"role": "system", "content": system_prompt})
        messages.append({"role": "user", "content": message})

        response = client.chat.completions.create(
            model=MODEL_NAME,
            messages=messages,
        )
        #print(f"\n\n####check_is_complete: last_message is {response_content}. response was {response}\n\n")
        do_more = 'yes' in response.choices[0].message.content.lower()
        #print(f"Do more steps need to be taken? {do_more} based on '{response_content}'.")
        return not do_more
    except Exception as e:
        print(e)
        return False


query = "Find a list of friends and send birthday greetings if today is a friend's birthday"
chain_message = "Find a list of friends and send birthday greetings if today is a friend's birthday"
messages = []
messages.append({"role": "system", "content": system_prompt,})
messages.append({"role": "user", "content": chain_message,})

while True:
    try:
        response = get_response(messages)
        response_message = response.choices[0].message
        response_content = response_message.content
        tool_calls = response_message.tool_calls
        #print(response_message)
        #print(response_content)
            
        is_complete = check_is_complete(query, response_content)
        if is_complete and tool_calls == None:
            break
        if not tool_calls:
            continue
        assistant_message = {"role": "assistant", 'tool_calls': tool_calls}
        messages.append(assistant_message)

        messages = run_tools(tool_calls, messages)
        next_question = "What step should I take next?"
        messages.append({"role": "user", 'content': next_question})
    except Exception as e:
        print(e)
        print(traceback.format_exc())
        pdb.set_trace()
        break

output looks like

Executing find_list_of_friends..
Function find_list_of_friends with arguments {} returned ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve'].
Executing find_todays_date..
Function find_todays_date with arguments {} returned 2024-01-22.
Executing find_birthdays_of_friends..
Function find_birthdays_of_friends with arguments {'friends_list': ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve']} returned [{'friend': 'Alice', 'date_of_birth': '1998-02-11'}, {'friend': 'Bob', 'date_of_birth': '1992-04-24'}, {'friend': 'Charlie', 'date_of_birth': '1995-09-14'}, {'friend': 'Diana', 'date_of_birth': '1991-08-22'}, {'friend': 'Eve', 'date_of_birth': '1996-03-23'}].
Executing send_birthday_greetings..
Sent birthday greeting to Bob born on 1992-04-24
Function send_birthday_greetings with arguments {'birthdays_list': [{'friend': 'Bob', 'date_of_birth': '1992-04-24'}], 'today': '2024-01-22'} returned Greetings sent.

Today’s date does not need to be a function (where an iteration can be thousands of context tokens). That can just be injected into the system prompt.

import time

struct_time_local = time.localtime()
struct_time_utc = time.gmtime()
local_time = time.strftime("Session start time (local): %Y-%m-%d %H:%M", struct_time_local)
utc_time = time.strftime("Session start time (UTC): %Y-%m-%d %H:%M", struct_time_utc)

system = [{"role": "system", "content": 'You are an AI assistant.'},
    {"role": "user", "content": 'As a connection debugger, you only reply with the date.'}]

system[0]['content'] += f"\n{local_time}"

The descriptions of functions and their parameters can be longer and multi-line.

It seems what you need is that if the user requests something fulfilled by the final function, it must state in the description where those parameters come from, and that the function cannot be used alone. Ambiguous fields can even have the AI making up stuff to send. I recommend:

“name”: “send_birthday_greetings”,
“description”: “Requires AI knowledge of today’s date and AI knowledge of friend list and friend birthdays. AI does not have pretrained knowledge of these events. Parameters must be obtained by function calls invoked immediately before.”,