Bug: Infinity Stream Loop with response_format

Regardless whether the special GPT-4 AI model dumps a whole bunch of nonsense repeating tokens or says “I have no idea what you want me to produce”, you can expect unsatisfactory output with that prompting.

The model can’t magically understand what type of json to produce without further specification, and training AI on some default would only degrade the quality of other specifications.

It looks like you want to chat with an AI. So let’s do a system message that is actually going to produce useful chat output you can parse and more demonstration, with full-on schema programming:

You are a helpful AI assistant.
Your output is to an API.
Response to user and metadata will be extracted from output json.
Except for tool calls, create only valid json complying to schema.

// json output schema
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "message": {
      "type": "object",
      "properties": {
        "assistant": {
          "type": "string",
          "description": "the response to the user",
          "example": "I am glad to help!"
        },
        "topic": {
          "type": "string",
          "description": "subject of recent discussion",
          "example": "baseball"
        },
        "user_mood": {
          "type": "string",
          "description": "user is happy, upset, etc",
          "example": "neutral"
        }
      },
      "required": [
        "assistant",
        "topic",
        "user_mood"
      ]
    },
    "length_of_conversation_turns": {
      "type": "number",
       "description": "total user and assistant conversation exchange turns"
    },
    "conversation_turns_on_topic": {
      "type": "number",
      "description": "number of recent turns engaging newest topic"
    }
  },
  "required": [
    "message",
    "length_of_conversation_turns",
    "conversation_turns_on_topic"
  ]
}

You will see that quality specification kind of makes the json model invoked by response format pointless.

(I also show the AI informing you how much you can lop off the conversation history)