Found a bug in GPT assistant which I am reporting here

API
1

I am using the below loop for an assistant with function calling . Sometimes the run never returns the status “requires_action”, it returns “completed”. My understanding is that function calling GPT assistant will return “requires_action” which I need to interpret to do the required function calling.

        while run.status != "requires_action":
            print(run.status)
            run = openai.beta.threads.runs.retrieve(
                thread_id=thread.id,
                run_id=run.id
            )

Below is a screenshot, after in_progress, goes to complete

image
image532×624 3.52 KB

2

If the user input is best answered by information from an external function that you provide as a tool, and the tool is properly described to show the AI how the function would be of utility in supplementing the answer the AI might give, then it would be called.

The AI model from its instructions, user prompt, and function description will call a function only when it is useful, such as a weather_today(location) that might be of utility for “what’s the forecast for Miami” answering.

The AI won’t just randomly or always use a function.

3

Ok, but I thought if I provide “type:function” to the GPT assistant, it will follow the life cycle of a function calling assistant. If it is not doing this, this will be very difficult to implement. I am doing some more digging and will share my findings

4

I think this is a bug
When it returns a status of “completed”, the output looks as below

#Run(id='run_7PwxFeLM3g22swyW26oNvQfd', assistant_id='asst_j7x9jBZSGmiLRvggElZ8l9Z4', cancelled_at=None, completed_at=1701024960, created_at=1701024957, expires_at=None, failed_at=None, file_ids=[], instructions='You are a helpful weather bot. You use the provided functions to answer questions', last_error=None, metadata={}, model='gpt-4-1106-preview', object='thread.run', required_action=None, started_at=1701024957, status='completed', thread_id='thread_P1IbW8PcP7j0eb3UHtKBJkwj', tools=[ToolAssistantToolsFunction(function=FunctionDefinition(name='getCurrentWeather', parameters={'type': 'object', 'properties': {'location': {'type': 'string', 'description': 'The city and state e.g. San Francisco, CA'}, 'unit': {'type': 'string', 'enum': ['c', 'f']}}, 'required': ['location', 'unit']}, description='Get the weather in location'), type='function')])

When it returns a status of “requires_action”, the output looks as below

#Run(id='run_8aCl6wiotKMBTcQpUtUgkTv4', assistant_id='asst_j7x9jBZSGmiLRvggElZ8l9Z4', cancelled_at=None, completed_at=None, created_at=1701025110, expires_at=1701025710, failed_at=None, file_ids=[], instructions='You are a helpful weather bot. You use the provided functions to answer questions', last_error=None, metadata={}, model='gpt-4-1106-preview', object='thread.run', required_action=RequiredAction(submit_tool_outputs=RequiredActionSubmitToolOutputs(tool_calls=[RequiredActionFunctionToolCall(id='call_jpKWGWmmHfbGYkZmRlzKxz8Q', function=Function(arguments='{"location":"CA","unit":"f"}', name='getCurrentWeather'), type='function')]), type='submit_tool_outputs'), started_at=1701025110, status='requires_action', thread_id='thread_TQTtwXKYnyrYjDj3xRN1hPwg', tools=[ToolAssistantToolsFunction(function=FunctionDefinition(name='getCurrentWeather', parameters={'type': 'object', 'properties': {'location': {'type': 'string', 'description': 'The city and state e.g. San Francisco, CA'}, 'unit': {'type': 'string', 'enum': ['c', 'f']}}, 'required': ['location', 'unit']}, description='Get the weather in location'), type='function')])

In both cases, it is identifying it as a function call, but the lifecycle and output is different. I think this is not how it was intended to work.

5

No. GPT will determine if a function should be called. It is not forced.

The first is the schema you have provided which needs to be used alongside each message for GPT to decide if it should be called.

The second is the function & parameters that GPT has created for you to run.

This is not a bug.

6

One of them is a bug, either the implementation or the documentation mentioned in this link for function calling assistants

https://platform.openai.com/docs/assistants/tools/function-calling

7

When you initiate a Run with a user Message that triggers the function

You can find the schema for the Run object here:

https://platform.openai.com/docs/api-reference/runs/object

8

I don’t know if this can help you, but something similar happened to me, and then I realized that I simply hadn’t passed the message on

`client.beta.threads.messages.create(

    thread_id=thread.id,
    role="user",
    content="the question is..."
    )`