I am using client.chat.completions.create() for my app and the response from completion.choices[0].message.content suddenly has format
{"messages": [{"role": "user", "content": "hello"}, {"role": "assistant", "content": "Hello! I'm your assistant, here to answer any questions you may have about our products and services. How can I help you today?"}]}
instead of just
Hello! I'm your assistant, here to answer any questions you may have about our products and services. How can I help you today?
The weird thing is that it works just fine yesterday and I did not make any change to my code. I understand that I can reformat the response to extract the text, but I am worried about the inconsistency of the output and this may cause problems in real production. Could you give some suggestion?
It would not be so much code, but rather, the model being used, and the actual messages that can replicate this. I trust that the extraction from the response is done correctly as described, and there is no actual object with “messages” as the only key with a list in code.
I said “hi” to gpt-4o-mini, it’s fine. Several others are “Hello! How can I assist you today”, so the language output shown about products and services has more behind it.
What is shown as response is a bit like a fine-tuning file. Any system message talk could get a particular model to decide to write in that fashion, but it also could be an API issue if it is constantly reproducible, but changes with model or with a temperature increase ('oops, the new guy stealthfully post-trained wrong).