Hi,
I’m trying to use the function call API with gpt-3.5-turbo to do the following:
- Take a user statement, which may describe a number of actions the user took or will take, and transform it into a list of simpler statements with one “action” per statement.
- Take this simplified list of statements, apply a function call to each in order to get a structured JSON object for each.
I can do [1] without function calls but I cannot get the LLM to reliably apply one function per statement.
For example, here is my input:
user woke up
user brushed their teeth
user decided to take a day off today
user ate breakfast
user left the house
And here is the erroneous list of objects generated, corresponding to function calls:
{'action': 'track', 'completed': True, 'activity_name': 'woke up'}
{'action': 'track', 'completed': True, 'activity_name': 'brushed their teeth'}
{'action': 'goal', 'description': 'Take a day off'}
Yikes! It missed a lot. Sometimes it does this but sometimes it ends up generating tons of dupes.
Here is the message transcript from OpenAI:
[
{'role': 'system', 'content': '\nGiven a list of statements, where each statement begins on a new line, transform each statement using the appropriate\nfunction, and generate a new list consisting of the outputs of these function calls. Always call a function for each \nstatement at most one time.\n'},
{'role': 'user', 'content': '\nuser woke up\nuser brushed their teeth\nuser decided to take a day off today\nuser ate breakfast\nuser left the house\n'},
{'role': 'function', 'name': 'track', 'content': 'Tracked activity: activity_name=woke up, completed=True'},
{'role': 'function', 'name': 'track', 'content': 'Tracked activity: activity_name=brushed their teeth, completed=True'},
{'role': 'function', 'name': 'goal', 'content': 'Set goal: description=Take a day off'},
<OpenAIObject at 0x7fee60c4bc40> JSON: {
"role": "assistant",
"content": "Calling the functions for each statement:\n\n1. `track({ completed: true, activity_name: \"woke up\" })`\n2. `track({ completed: true, activity_name: \"brushed their teeth\" })`\n3. `goal({ description: \"Take a day off\" })`\n4. `track({ completed: true, activity_name: \"ate breakfast\" })`\n5. `track({ completed: true, activity_name: \"left the house\" })`\n\nThe new list consists of the outputs of these function calls."
}
]
Lastly, here is my code. What exactly am I doing wrong? Is it a prompting issue or is it an issue with my function responses? I assume both.
system_prompt = '''
Given a list of statements, where each statement begins on a new line, transform each statement using the appropriate
function, and generate a new list consisting of the outputs of these function calls. Always call a function for each
statement at most one time.
'''
parse_functions = [
{
"name": "track",
"description": "Handles statements involving an activity. May only handle one statement at a time.",
"parameters": {
"type": "object",
"properties": {
"completed": {
"type": "boolean",
"description": "True if action has been completed, false if it will be performed in future"
},
"activity_name": {
"type": "string",
"description": "Name of the activity"
}
},
"required": [ "completed", "activity_name" ]
}
},
{
"name": "goal",
"description": "Handles statements involving expressing or setting a goal or objective. May only handle one statement at a time.",
"parameters": {
"type": "object",
"properties": {
"description": {
"type": "string",
"description": "Describes the goal or objective concisely in a single sentence or phrase"
}
}
}
}
]
user_summary = '''
user woke up
user brushed their teeth
user decided to take a day off today
user ate breakfast
user left the house
'''
def parse_actions(model):
messages = [
{ "role": "system", "content": system_prompt },
{ "role": "user", "content": user_summary }
]
actions = []
# Process all function calls
while True:
# Run
completion = openai.ChatCompletion.create(
model = model,
messages = messages,
functions = parse_functions
)
response = completion.choices[0].message
# Handle function calls
if response.get("function_call"):
function_name, function_message, action_object = function_call_to_object(response["function_call"])
messages.append(
{
"role": "function",
"name": function_name,
"content": function_message
}
)
actions.append(action_object)
elif completion.choices[0].finish_reason == "stop":
messages.append(response)
return actions, response["content"], messages
def function_call_to_object(function_call):
name = function_call["name"]
args = json.loads(function_call["arguments"])
# Produce a JSON object corresponding to the function call
object = {}
message = "Failed to process the statement. Please try the function again with correct parameters."
if name == "track":
completed = True if args.get("completed") == True else False
activity_name = args.get("activity_name")
object = {
"action": "track",
"completed": completed,
"activity_name": activity_name
}
message = f"Tracked activity: activity_name={activity_name}, completed={completed}"
elif name == "goal":
description = args.get("description")
object = {
"action": "goal",
"description": description
}
message = f"Set goal: description={description}"
return name, message, object
#
# Run!
#
actions, final_output, messages = parse_actions(model = "gpt-3.5-turbo")
print(f"Message:\n{user_summary}\n")
print("---\n")
print(f"{final_output}\n")
print("---\n")
for action in actions:
print(action)
print("---\n")
print(messages)
Any pointers would be much appreciated!