Can't mix vision and function calls in gpt-4-turbo-2024-04-09

API Bugs
1

When using chat completions with gpt-4-turbo-2024-04-09, it seems that while function calls and vision are supported, they can’t be used together in the same chat.

The behavior can be reproduced with the files here: GitHub - spatialpixel/gpt-4-turbo-2024-04-09: Possible bug in gpt-4-turbo-2024-04-09

These files have the same code, but with messages regarding vision or function calls commented out in the working files.

When running vision-function-fails.js, this results in a 400 error.

/Users/awmartin/gpt-4-turbo-bug/node_modules/openai/error.js:44
            return new BadRequestError(status, error, message, headers);
                   ^

BadRequestError: 400 Invalid chat format. Unexpected keys in messages.
    at APIError.generate (/Users/awmartin/gpt-4-turbo-bug/node_modules/openai/error.js:44:20)
    at OpenAI.makeStatusError (/Users/awmartin/gpt-4-turbo-bug/node_modules/openai/core.js:263:33)
    at OpenAI.makeRequest (/Users/awmartin/gpt-4-turbo-bug/node_modules/openai/core.js:306:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async main (/Users/awmartin/gpt-4-turbo-bug/vision-function-fails.js:70:22) {
  status: 400,
  headers: {
    'alt-svc': 'h3=":443"; ma=86400',
    'cf-cache-status': 'DYNAMIC',
    'cf-ray': '871e73295a06c472-EWR',
    connection: 'keep-alive',
    'content-length': '161',
    'content-type': 'application/json',
    date: 'Tue, 09 Apr 2024 23:59:15 GMT',
    'openai-organization': 'user-jiekerw6qofpqy89ta7h7wfr',
    'openai-processing-ms': '9',
    'openai-version': '2020-10-01',
    server: 'cloudflare',
    'set-cookie': '__cf_bm=5oN_P8bTEUuOy_O19rgs07yR3LLK.gsPf5uRz1zyYB8-1712707155-1.0.1.1-Ec77yqfzPi1lvOMccYvXKvD61SKbupqtCgufpGljxFh2CADHqqg.0I5XJy6bRs2AUVb_Y1y1nU9rp1iPhlRLKA; path=/; expires=Wed, 10-Apr-24 00:29:15 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=KIhipQW09KHdi6Q26hIRFNMRUJr1TptGo2h67VF3xRc-1712707155511-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None',
    'strict-transport-security': 'max-age=15724800; includeSubDomains',
    'x-ratelimit-limit-input-images': '500',
    'x-ratelimit-limit-requests': '500',
    'x-ratelimit-limit-tokens': '450000',
    'x-ratelimit-remaining-input-images': '499',
    'x-ratelimit-remaining-requests': '499',
    'x-ratelimit-remaining-tokens': '449129',
    'x-ratelimit-reset-input-images': '120ms',
    'x-ratelimit-reset-requests': '120ms',
    'x-ratelimit-reset-tokens': '116ms',
    'x-request-id': 'req_eae793546c462268bc86d495e0407f98'
  },
  error: {
    message: 'Invalid chat format. Unexpected keys in messages.',
    type: 'invalid_request_error',
    param: null,
    code: null
  },
  code: null,
  param: null,
  type: 'invalid_request_error'
}

Node.js v20.9.0
2 Likes
2

It’s working this afternoon without any changes!