Response JSON is cut off with JSON mode enabled

Hi, Community

I am using gpt-3.5-turbo-1106 with enabled JSON mode to retrieve a quote JSON with text, and the author’s first, and last name. But occasionally it fails to complete the JSON output, even though FinishReason=stop.

I receive such string in response:
{\n \"SingleQuote\":\n {\n \"QuoteText\": \"The best way to find yourself is to lose yourself in the service of others.\",\n \"AuthorFirstName\": \"Mahatma\",\n \"AuthorLastName\": \"

Seems like the problem is not with MaxTokens: PromptTokens = 900, CompletionTokens = 47, MaxTokens=100.

Ofcourse, I state about JSON in the system message + provide a response example in JSON format.

System message:
You are an AI system that simulates expertise in human psychology.\nYou receive a person description as input data.\nAdditionally as input you receive a task that describes what kind of quotes to provide.\nAfter carefully analyzing the person's description, systematically evaluate and identify the most appropriate quotes that align with the task by thoroughly considering each step of the thought process.\nIn addition to the input, you are given the names of authors of quotes that you MUST exclude from your response.\nYour only objective is to provide a quote.\nYou MUST provide only 1 quote.\nYou respond in json format only.\nEach quote in the response MUST have quote text, author first name, and author last name.\nExample of your response in JSON format:\n{\n \"SingleQuote\":\n {\n \"QuoteText\": \"$QUOTE_TEXT\",\n \"AuthorFirstName\": \"$QUOTE_TEXT_AUTHOR_FIRST_NAME\",\n \"AuthorLastName\": \"$QUOTE_TEXT_AUTHOR_LAST_NAME\"\n }\n}\n

User message:
\nPerson description:\n```\nA person has a clear desire for rest and relaxation, avoiding conflict, and longing for security. They seem to have relegated their aspirations and focus instead into creating a comfortable and safe environment, as a response to their need to recover from the stress they have been experiencing. This individual seems methodical and independent in their approach to life, and they value respect, recognition, and understanding from their peers, which implies a social aspect to their sense of self-worth.\n\n Despite appearing to be egocentric and requiring validation, it seems they also feel easily wounded when support is not forthcoming. A heavy reliance may be placed upon personal convictions and plans, suggesting an internal, intensely personal set of goals or standards that they measure themselves against. However, they simultaneously seek out approval and endorsement from others, indicating a tension between self-reliance and the need for external validation.\n\nThis person is currently dealing with stress rooted in disappointment and a protective stance towards further failures. Their insistence for freedom of action and rejection of control, except when it aligns with their own terms, further points to a strong desire for autonomy. This may also be a reflection of a fear of loss regarding status or the potential inability to meet their standards.\n\nThe individual is struggling with perceived limitations and the fear that new aspirations may prove futile. They demonstrate an avid desire to be recognized and to interact meaningfully with others. It appears that disappointments and a sense of emptiness in relationships may have fostered a sense of alienation, prompting an intensive drive to achieve their personal goals. The quest for security is prominent, and they seem to wish for a scenario where they no longer face demands or pressures. Anxiety from failed expectations and the dread of further disappointment has led to both an internal turning-inward and a tangible sense of restlessness and gloom, as noted in their depressed mien.\n\nModerate anxiety has been affecting their daily activities, indicating that their psychological challenges are substantial enough to impinge upon their daily functioning. It appears that this level of stress is having a tangible impact, likely affecting decision-making, interactions with others, and overall well-being.\n```\nQuotes main task: \n\nTo help the person from the description to feel 'Happiness' through a quote. Happiness, a complex and multi-dimensional emotion, involves several facets. It is influenced by neurochemicals like dopamine, serotonin, oxytocin, and endorphins. It's shaped by cognitive processes, such as optimism, gratitude, and mindfulness, as well as behavioral aspects including exercise, sleep, and social interaction. Psychological well-being, encompassing self-esteem, resilience, autonomy, and a sense of purpose, plays a significant role in happiness. It's also impacted by social relationships, cultural norms, environmental conditions, economic stability, and spiritual beliefs. The state of one's physical and mental health can greatly affect happiness. Finally, happiness can be both a fleeting emotional response and a more stable long-term state, and often involves a sense of existential meaning and purpose. One of the aspects of the feeling is 'Spiritual Perspective'. Description of this aspect is: For many people, spiritual beliefs and practices, including religious affiliation, meditation, and a sense of connection to something greater than oneself, are vital components of happiness.Following all of that Quotes about the spiritual aspect of happiness might inspire a sense of connection to something larger than oneself. They could encourage religious or spiritual exploration, meditation, mindfulness, or a sense of awe and wonder.\n\n

LogitBias: {"39":-10,"67391":-10,"79244":-10,"71":-10,"23871":-10,"33947":-10,"24241":-10,"57621":-10,"6380":-10,"35":-20,"2234":-20,"88":-20,"423":-20,"43":-20,"3105":-20,"83659":-20,"32":-20,"1557":-20,"1395":-20,"362":-20,"81101":-20,"326":-20,"31224":-20,"906":-20,"21114":-20,"56834":-20,"34":-20,"1923":-20,"61157":-20,"64373":-20,"8161":-20,"87731":-20,"4418":-20,"49492":-20,"268":-20,"63592":-20,"49":-20,"1136":-20,"27591":-20,"35175":-20,"44":-20,"38221":-20,"582":-20,"24789":-20,"301":-20,"38":-20,"438":-20,"6151":-20,"48994":-20,"47":-20,"402":-20,"2423":-20,"43856":-20,"50135":-20,"61887":-20,"9734":-20,"27588":-20,"5735":-20,"34422":-20,"82265":-20,"817":-20,"16852":-20,"543":-20,"276":-20,"682":-20,"79":-20,"4748":-20,"3273":-20,"68":-20,"14804":-20,"40430":-20,"88315":-20,"18735":-20,"9641":-20,"54":-20,"1147":-20,"331":-20,"468":-20,"71524":-20,"17527":-20,"42":-20,"370":-20,"266":-20,"11419":-20,"6258":-20,"41536":-20,"2308":-20,"1412":-20,"3703":-20,"41767":-20,"23946":-20}

Temperature: 1.2
MaxTokens: 100

What am I doing wrong? How to fix that?

Hi Artem

The problem is most probably in temperature. 1.2 is too high even for creative tasks that require some consistency, let alone generating json. 0 would be best for most of such cases.

Hi @TonyAIChamp

I intentionally set temperature 1.2 to retrieve different, yet suitable, quotes.
Isn’t set temperature to 0 will get me the same quote each time?

It will be more stable yes. But at the same time I don’t believe you can get stable json output with such high t.

So, the issue was about having a token from LogitBais in the output (it should be next).

Seems like this is a bug in ChatGPT. It would be better to receive an error, rather than incomplete JSON.