Buggy behaviour with responseformat json_object with functions available

Hi,

I have a problem with function calls that should never happen when I use response format json object, especially with gpt-4o-2024-05-13 (but occationally with gpt-4o-2024-08-06 as well).


Steps to reproduce in playground:

  • Add any function, for example the get_weather sample function
  • Set Response format to json_object
  • Leave system message empty
  • Set user message to any valid request for json, for example: Return an empty json object, like “{}”
  • Run

Expected outcome
{}

Actual outcome
One or multiple tool calls.


It also keeps on calling function(s) again an again on tool responses, so infinite looping is quite possible here if you have not protected yourself against that.

I guess the workaround/right way is to switch to gpt-4o-2024-08-06 and use json_schema?

But the error seems quite severe to be in the 4o default model. What are your thoughts?