Retrieval and JSON mode, using together

[marco2]

I was wondering, why is JSON Mode AND retrieval (or code-interpreter) not available? I can choose one or the other, but not both.

[nikunj]

Thanks for the feedback but this is unfortunately not supported yet. We have a model inference limitation to make tool calls work with JSON mode but we are working on fixing that and will enable this functionality as soon as that’s ready!

[_j]

I gave another topic with the same question some insights just an hour before this was posited again here.

`Json` response type in Assistant

Code interpreter is powered by a tool called python, not a tool for all functions The python tool internal to assistants does not take output with arguments in json The response_format would affect all AI generation after the AI emits tokens for tools or to decide to respond to the user. The result would be python receiving a loop of tabs or linefeeds until the max_tokens is hit, getting an error return, and then trying again multiple times.

We can only infer how the JSON mode actually works - by producing a fixed response, and incrementing max_token one at a time, you can see when this token-run based method takes over and replaces the prior few logits with brackets, plus linefeed and tab characters very unlikely otherwise (should have been trained/biased on JSON without white space)