Function api returns value that are not in enum?

API
1

Hey guys,
a question about functions.
There is a function signature and a response below.
Nothing fancy - I do some test with a news api.
The message is what I get back from the openai API
My prompt was “What’s new in politics?”
and I get a category “politics”
however in the function signature category is an enum - and politics is not part of that enum. I expected that gpt would cleverly map that to the closest fitting category ?
The way it is now it causes a problem because the api behind it does only except values that are in the enum.

What am I missing ?

MESSAGE: {
  "content": null,
  "function_call": {
    "arguments": "{\n  \"category\": \"politics\"\n}",
    "name": "get_top_headlines_stories"
  },
  "role": "assistant"
}

top_headlines_signature = {
      "name": "get_top_headlines_stories",
      "description": "This function provides live top and breaking headlines for a country and specific category. Articles are sorted by the earliest date published first.",
      "parameters": {
        "type": "object",
        "properties": {
          "country": {
            "type": "string",
            "enum": ["de", "us", "ru", "gb", "fr", "it", "es"],
            "description": "The Country of the news source. Possible options: de, us, ru, gb, fr, it, es"
          },
          "category": {
            "type": "string",
            #options: business entertainment general health science sports technology
            "enum": ["business", "entertainment", "general", "health", "science", "sports", "technology"],
            "description": "The Category of the news headline. Possible options: business, entertainment, general, health, science, sports, technology"
          }
        },
        "required": ["category"]
      }
    }
2

You’re missing that it’s a language model that predicts text based on input and throws variance into the answers with temperature and sampling.

That OpenAI is overpromising something that is just a behavior guideline that can be overridden by the weight of a more appealing string of tokens that the user is allowed to place for themselves.

That unlike actual code, I can make it dump out and repeat back functions and command AI to put anything into your function that I want.

About the only thing with tuning good enough that you can’t jailbreak while still having a functioning chatbot is to make it not apologize.

3

You can either validate the return value of function call before sending to the api or handle it in the api.

Then if an unexpected value is detected, you can directly send back a message or run it in the final chat API call. Using the latter, part of your messages will have this:

{ role: "function", name: "get_top_headlines_stories", content: JSON.stringify({ error: "Invalid category" })}

The Chat API will send a clarifying response.

2 Likes
4

thanks. I know the underlying principles - I was just wondering if there was something in using the api I was missing.
But I agree with you: its astonishing how tempting it is to think of it as a deterministic programming machine :slight_smile:

5

Hey
that sounds interesting … I will try that. thanks.

6

hey @supershaneski

that is really interesting.
simmilar to what you suggested I just gave back the message in the exception and then I get back what is shown below:
still not “general” as in the enum. but interesting

this is the code snippet


try:
                response_message = get_top_headlines(country, topic)
            except Exception as e:
                messages.append({
                    "role": "function",
                    "content": json.dumps({"error": str(e)})
            })

Here are some top headlines from various categories:

  1. Business: “Dollar firm ahead of Powell testimony, sterling up on hot inflation” - Reuters Read more
  2. Entertainment: “Hero Xtreme 125 Spied For The First Time - Sporty & Affordable” - GaadiWaadi.com Read more
  3. Technology: “Amazon Prime Day 2023 will take place on July 11th and 12th” - The VergeRead more
  4. Business: “HDFC Bank gains as Goldman Sachs reinstates ‘buy’ call, sees 21% upside” - Moneycontrol Read more
  5. Business: “Market LIVE Updates: Sensex hits all-time high, Nifty above 18,850; IT, auto, realty gain” - Moneycontrol Read more
7

hm.
I have this code now.
Interestingly he insists on the category politics - runs through the same error a couple of times and then comes back with:
I’m sorry, but I couldn’t find any news in the politics category.
“Could you please specify a different category?”

try:
                response_message = get_top_headlines(country, topic)
                messages.append({"role": "function", "content": response_message, "name": "get_top_headlines_stories"})
       
            except Exception as e:
                print("Error: " + str(e))
                messages.append({
                    "role": "function",
                    "name": "get_top_headlines_stories",
                    "content": "error: " + str(e)
                    })