Hey Champs Anyone noticed that responses in json are much shorter than without it? Here are 2 examples of completions: ▶ plain answer ▶ json answer These completions were received from the executions in the same model (gpt-3.5-turbo-1106) with the same parameters (“temperature”: 0, "response_format…

Ok, that’s interesting, I actually haven’t noticed this kind of behavior before. Would be great if you could provide the entire prompt, interesting to check this out.

I want to do a bigger testing, but my guess is that it will work with any prompt. Working hypothesis is that trying to change the format of the output significantly affects the quality of following instructions.

Ok, I haven’t noticed it myself, update on what you come up with. Just something to consider: length isn’t necessarily quality :sweat_smile:

In my opinion it’s expected, given how most json objects don’t have lengthy values.

That is a valid point and I would consider it an explanation if not for this Json responses in gpt-3.5-turbo-1106 much shorter than without json? - #3 by TonyAIChamp

Obviously it is not, generally speaking, but in my specific case it is one of the factors of the quality.

Hi @TonyAIChamp - the way I solved this is to have a two-step conversation with the model. I first ask for the result for NLP response and then do a new message with the test “as JSON”. You may give an example in your system prompt for the JSON to make sure that the output conforms. Code looks som…

The model may also be preconditioned to the fact that JSON fields in its training data may not have been too big. That was my first guess. If you really really need the JSON with a big response chunk, then maybe call it normally without JSON, and then just add the big response to your JSON manuall…

Thank you for the idea, Greg! I ended up doing also 2-step, but a bit different: I first get a normal response from the model, and then ask a model in a separate conversation to give me the status of the conversation in json format (feeding it the main conversation and providing criteria for settin…

Thank, Curt! As mentioned above, this is a really great guess. The problem with it as a hypothesis for the reason of this specific behaviour is that it spans outside of json output: Json responses in gpt-3.5-turbo-1106 much shorter than without json? - #3 by TonyAIChamp

Json responses in gpt-3.5-turbo-1106 much shorter than without json?

API

TonyAIChamp December 6, 2023, 1:18am 27

@curt.kennedy I just did this test, same results as expected.

JSON Mode with GPT-4 turbo stops after 1050 token

Topic		Replies	Views
Davinci still seems like the gold standard, compared to turbo API	23	4983	April 21, 2023
JSON Mode with GPT-4 turbo stops after 1050 token Bugs api , gpt-4-turbo	26	5179	February 5, 2024
Why is ChatGPT so moody, refusing to answer the exact same question it answered previously? API	14	5575	March 27, 2023
Chat GPT4 1106 vs ChatGPT 4: Impressive drop in quality API gpt-4 , chatgpt	27	15939	February 14, 2024
Gpt-3-5-turbo-1106 either timeout or gives radically different result from gpt-3.5-turbo-16k API gpt-35-turbo	9	3544	December 4, 2023

Json responses in gpt-3.5-turbo-1106 much shorter than without json?

Related topics