Have functions changed? I keep getting non-sense responses

Using the below code as input just gives me “New York” as output, for some reason it seems like functions have gotten materially worse, I cant for the life of me get any other response that the input - any ideas?

f_s_location="New York"
f_dict_gpt_input_json_structure_city = {
    'model': "gpt-3.5-turbo",
    'temperature': 0.2,
    "messages": [
        {"role": "user", "content": "What is the population of the following city: "+str(f_s_location)}
    ],
    "function_call":{"name": "population_function"},
    "functions": [
    {
    "name": "population_function",
    "description": "Estimate population of a city",
    "parameters": {
        "type": "object",
        "properties": {
            "city": {
            "type": "string",
            "description": "population"
            },
        },
        "required": ["city"]
    }
    }
]
}

o_gpt_response = requests.post(
    'https://api.openai.com/v1/chat/completions',
    headers=dict_gpt_headers,
    json=f_dict_gpt_input_json_structure_city,
    timeout=10
)
print(o_gpt_response.text)

(sorry for not addressing the original question, I’m not aware of any direct changes)

Perhaps the model believes that calling this function will estimate the population and return a number.

Perhaps make the type a number, or update the description a bit?

The below just returns 0:

f_s_location="New York"
f_dict_gpt_input_json_structure_city = {
    'model': "gpt-3.5-turbo",
    'temperature': 0.2,
    "messages": [
        {"role": "user", "content": "City: "+str(f_s_location)},
        {"role": "system", "content": "Estimate the population of the input city"}
    ],
    "function_call":{"name": "population_function"},
    "functions": [
    {
    "name": "population_function",
    "description": "Estimate population of a city",
    "parameters": {
        "type": "object",
        "properties": {
            "population_city": {
            "type": "number",
            "description": "Population of a city"
            },
        },
        "required": ["population_city"]
    }
    }
]
}

o_gpt_response = requests.post(
    'https://api.openai.com/v1/chat/completions',
    headers=dict_gpt_headers,
    json=f_dict_gpt_input_json_structure_city,
    timeout=10
)
print(o_gpt_response.text)

You have given the AI a function the purpose of which appears to be to submit a city name and received back a population estimate. The AI will then use the return value to enhance its answering capability.

If that is not what you think you did, you need to re-examine your understanding of functions. The function call ability is not for having the AI write its output to the user.

Thats actually hilarious, I have been using functions wrong and primarily to force GPT into giving me a singluar answer, which has worked well so far.

How would you go about preventing verbose answers if you dont use functions? Because it tends to give answers like “The population of New York is 10 million people” instead of “10 million”?

1 Like

gpt-3.5-turbo-instruct

image

Unfortunately that is not good enough - functions even when used wrong have a very good ability to return exact numbers, categories, etc. especially when using enum

Then you’ll need to create some simulation function to obtain the output you want, like “displays formatted city population in user interface”, and a return value that reports success if this is a chatbot and not a function itself.

Or another trick “required: submits proposed answer to safety moderation system”

That sounds really cool! Like similarly to the below - although that still just returns “København”? (To make it easier I am just trying to translate a city name into English)

f_s_location="København"
f_dict_gpt_input_json_structure = {
    'model': s_gpt_model,
    "temperature": 0.2,
    "messages": [
        {"role": "user", "content": "What is the country of "+str(f_s_location)}
    ],
    "function_call":{"name": "categorisation_function"},
    "functions": [
    {
    "name": "categorisation_function",
    "description": "Displays country names of cities",
    "parameters": {
        "type": "object",
        "properties": {
            "city": {
            "type": "string",
            "description": "English name of the city"
            },          
        },
        "required": ["city"]
    }
    }
]
}

Where would you put the “required: submits proposed answer to safety moderation system”

If this is a single purpose like you describe, lets whip up some function to make the AI output translated city requests to it.

{
    "name": "city_name_output_in_english",
    "description": "Use for returning city names which AI translates to English",
    "parameters": {
        "type": "object",
        "properties": {
            "original_name": {
                "type": "string",
                "description": "Native or foreign name of city"
            },
            "english_name": {
                "type": "string",
                "description": "English name of city"
                },
        },
        "required": ["original_name", "english_name"],
    },
}

And then use it in practice:

import openai
import json
import os
openai.api_key = os.getenv("OPENAI_API_KEY")

def show_city_translation(city_dict):
    return (f"In English, {city_dict['original_name']}"
            f" translates to {city_dict['english_name']}")

def city_chat():
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        max_tokens=500,
        messages=[
            {
            "role": "system",
            "content": 'You are Bob the friendly AI.'
            },
            {
            "role": "user",
            "content": 'translate: København'
            }
            ],
        functions=[
            {
                "name": "city_name_output_in_english",
                "description": "Use for returning city names which AI translates to English",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "original_name": {
                            "type": "string",
                            "description": "Native or foreign name of city"
                        },
                        "english_name": {
                            "type": "string",
                            "description": "English name of city"
                            },
                    },
                    "required": ["original_name", "english_name"],
                },
            }
        ],
        function_call="auto",
    )
    message = response["choices"][0]["message"]
    # check for a function call
    if message.get("function_call"):
        function_name = message["function_call"]["name"]
        function_args = json.loads(message["function_call"]["arguments"])
        if function_name == "city_name_output_in_english":
            print(show_city_translation(function_args))
    else:
        # do normal chatbot stuff
        pass

result = city_chat()

In English, København translates to Copenhagen