Any issue with gpt-4o model recently relating to tool calls?

API Bugs
1

i noticed some inconsistency in the tool call recently,

{
        "type": "function",
        "function": {
            "name": "verify_user",
            "description": "requires user to input their account number",
            "parameters": {
                "type": "object",
                "properties": {
                    "account_number": {
                        "description": "the user's account number",
                        "type": "string"
                    }
                },
                "required": [
                    "account_number"
                ]
            }
        }

until yesterday, the model always ask the user for their account number, now, it assume that it already know the account number and filling a default either:

function args {‘account_number’: ‘xxxxxx’}
or
function args {‘account_number’: ‘123456’}

i tried gpt-4-turbo to test and seems to be working fine

2

This function call in Chat Completion or Assistant APIs? If you’re using Chat Completion, use this is System Instruction:

Don’t make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.

If you are using Assistant APIs, put this above in ‘additional instruction’ of your ‘run’.

Let me know if this doesn’t fix your issue. We can try more things.

3

Yes definitely, especially gpt-4o-mini with Structured Outputs with Assistants. Increasing degradation over the past week. 4o seems more stable.

4

I have the same issue, I assume. When i started feeding to the chat model some data, it understands it, but runs the tool several times and then stops due to large number of agent execution. It fetches ok though from the chat itself. I am using langchain. But when i changed the model from o to turbo - all worked. The 4o still is handling prompts ok and provides the desired more less output via promps, but fails with functions. I think it is a bug on OpenAI site, they claim to have 4o as the more stable and the latest, but in reality turbo works better and way more stable. This issue I have seen before from time to time, but from 30.08 it is consistent and even forced me to re-write the code today 2 hours before presentation. Is there any intentional degradation of the model?

1 Like
5

Uodate. It started working with august version of gpt-4o. it would be nice for OpenAI to notify about the changes of current gpt-4o version

7

I’m using chat completion, and yes my prompt always has that instruction since, basically it was 100% working until yesterday with no code changes.

1 Like
8

I was looking at this too, I saw the structure changed, I updated to this structure but still the same issue, unless I changed to Turbo then it works however other issue also exist in turbo i.e. function to handover to human suddenly doesn’t get called.

10

Update, I switched to gpt-4o-2024-08-06 model and seems to be stable after several testing. Hopefully Openai fixes this issue in the current model.

1 Like