There’s a thread here:
the gist is that the JSON corset probably interdicts the file search/ special function calling
That said, I’ve never really had an issue with generating JSON without JSON mode: in the last message, just pass a proper schema, tell it to only respond in JSON, and tell it to start its response with {.