Using parameters with v3.5 (0613) and enum values and GPT uses values not in the enum definition

API
1

I’m trying to use the new parameters variable to pass along functions for chatGPT to use, but the values I pass for the "sort " property using “enum” are being overwritten, and I get values that were not in the enum array that I pass.
Example:

                'name'=> 'getOrders',
                'description'=> 'Get all Wix orders in an unsorted array',
                'parameters'=> [
                    'type'=> 'object',
                    'properties'=> [
                        'returnFields'=> [
                            'type'=> 'string',
                            'enum' =>  ['id', 'number' ,'dateCreated' ,'buyerInfo.id' ,'buyerInfo.identityType' ,'buyerInfo.firstName' ,'buyerInfo.lastName' ,'buyerInfo.phone' ,'buyerInfo.email']
                        ],
                        'sort' => [
                            'type' => 'string',
                            'enum' => ['dateCreated','lastUpdated','number']
                        ]
                    ],
                    'required'=> ['returnFields']
                ]

The response I get is


Response: Array
(
    [role] => assistant
    [content] => 
    [function_call] => Array
        (
            [name] => getOrders
            [arguments] => {
  "returnFields": "totals.total,buyerInfo.email,buyerInfo.fullName",
  "sort": "totals.total"
}
        )

)```
Both buyerInfo.fullName and totals.total are not a part of the definition enum, but they end up in the response.
Anyone else experience something like that (or otherwise sees what I'm doing wrong)
Thanks!
2

What kind of temperature setting do you use?

For this complicated function call, I would suggest a setting of 0.0 - 0.2 to work properly.
It is also important, that in the users request, all data is clearly available to call the function or GPT starts to fabricate.

Another problem I see, is that the description fields for the parameters are missing, which are important and that the parameters in the enums are of mixed semantics. This could lead to confusions.

1 Like
3

Thanks, I haven’t played with temperature yet.
In one of the examples from openAI they show only enum field and no description.
I actually switched to use description instead of enum with “Supported values: xxxxx” which seems to work better.

4

I tried with temperature 0 and it still changes some field names and tries to sort with a value that’s different than the allowed ones.
Open to any ideas.
Anyone else experiencing this too?

5

I’m experiencing this with the parameter “language” - openAI omits it, even though it is required.

I pass the following:
[{‘name’: ‘getNews’,
‘description’: ‘News API function’,
‘parameters’: {
‘type’: ‘object’,
‘properties’: {
‘q’: {‘description’: ‘Query to return news stories’, ‘type’: ‘string’},
‘language’: {‘default’: ‘en’, ‘description’: ‘Language of News’, ‘enum’: [‘en’, ‘es’], ‘type’: ‘string’},
‘sortBy’: {‘description’: ‘Sort By item’, ‘type’: ‘string’},
‘pageSize’: {‘description’: ‘Page Size’, ‘type’: ‘integer’}}},
‘required’: [‘q’, ‘language’, ‘pageSize’]}
]

Note that q, language and pageSize are all required.

I receive the following back consistently:

“choices”: [
{
“index”: 0,
“message”: {
“role”: “assistant”,
“content”: null,
“function_call”: {
“name”: “getNews”,
“arguments”: “{\n "q": "US Economic News",\n "pageSize": 5\n}”
}
},
“finish_reason”: “function_call”
}
],

When I shorten “language” to “lang”, it will work, however, that’s not instilling a lot of confidence in chatGPT being able to deliver accurate parameters in a function_call.

6

Update- it appears that in my case, setting the temperature to 0 seemed to work for the above case, however, when I added another variable, it wouldn’t display it even though it was required (granted, I’m not following chatGPT’s format exactly, but pretty close):

[{‘name’: ‘getNews’, ‘description’: ‘News API function’, ‘parameters’: {‘type’: ‘object’, ‘properties’: {‘module’: {‘description’: ‘Python Module to Call to run function’, ‘type’: ‘string’, ‘default’: ‘functions.newsapi_org’}, ‘q’: {‘description’: ‘Query to return news stories’, ‘type’: ‘string’}, ‘language’: {‘description’: ‘Language of News’, ‘enum’: [‘en’, ‘es’], ‘default’: ‘en’, ‘type’: ‘string’}, ‘sortBy’: {‘description’: ‘Sort By item’, ‘type’: ‘string’}, ‘pageSize’: {‘description’: ‘Page Size’, ‘type’: ‘integer’}}}, ‘required’: [‘module’, ‘q’, ‘language’, ‘pageSize’]}]

Reply from chatcompletion:

“choices”: [
{
“index”: 0,
“message”: {
“role”: “assistant”,
“content”: null,
“function_call”: {
“name”: “getNews”,
“arguments”: “{\n "q": "US Economic News",\n "language": "en",\n "pageSize": 5\n}”
}
},
“finish_reason”: “function_call”
}
],

And yes, I understand the gaping security flaws with this, and yes, I will have a dictionary of “allowable” modules and functions to validate against prior to running code from chatGPT. :slight_smile:

7

You can also try to annotate the type keyword to the enum. Maybe GPT can infer it easier.
But using a low temp might be often the simplest solution.

8

What do you mean by this?
Temp 0 and I still get the same results: made up fields

9

Any update on this? Having the same issue! Enum doesn’t work and values are being made up…

10

Unfortunately no.
I’m waiting to get v4 access in hopes it knows to read parameters

11

I run into same issue, the way I resolve the issue is by prefixing the field name into the enum value.
Ex,

'properties'=> [
        'field1'=> [
            'type'=> 'string',
            'enum' =>  ['field1_id', 'field1_value' ,'field1_date']
        ]
    ],

The assumption is it can help boost the correlation between enum value to the field rather than normal/random words.

3 Likes
12

Thanks @MatrixStone for sharing! I’m using this now and its working in most cases. Found its not perfect and it still makes up values in some cases, but at least it includes the prefix… :sweat_smile:

1 Like
13

@tjh100 Wow, that did it. Thanks!

Even adding additional " around my values fixes the issue. E.g.

'properties'=> [
        'field1'=> [
            'type'=> 'string',
            'enum' =>  ['"id"', '"value"' ,'"date"']
        ]
    ],

This is on gpt-3.5-turbo-16k. I’d be interested to hear if anyone else finds the same.