Any tips on prompting Assistant API to return the entire response in JSON?

API
1

I’ve been getting mixed results with how the Assistant API is returning responses when asked to output it’s entire response in JSON format. Occasionally, it will do just that but then, almost randomly, output a detailed response in standard text and then just a summary in JSON at the end. Pretty sure I’m either not prompting it correctly, or I need to be more specific in the Assistant instructions. Here’s an example prompt:

Analyze the text and provide the following: 1) A summary of the text. 2) Word count 3) Areas for improvement. Output your entire response in JSON format with fields: “An ID I pass to it”, “Summary”, “Word Count”.

Any guidance on correctly asking it to output in JSON format consistently is greatly appreciated!

2

AI can’t count words well. A simple function easily can. If it were to work, you’d have to be clearer about what words are being counted.

The correct JSON term is “key” for the data name, and “value” for data. That allows clear communication with the AI.

Extremely explicit definition? How about this schema I constructed for another forum answer, not only exacting and demonstrative, but a form that can itself validate the AI output.

The example schema shows the AI any user-directed response must also be only in the container (and don’t expect the last numbers to be faithfully answered).

You are a helpful AI assistant.
Your output is to an API.
Response to user and metadata will be extracted from output json.
Except for tool calls, create only valid json complying to schema.

// json output schema
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "message": {
      "type": "object",
      "properties": {
        "assistant": {
          "type": "string",
          "description": "the response to the user",
          "example": "I am glad to help!"
        },
        "topic": {
          "type": "string",
          "description": "subject of recent discussion",
          "example": "baseball"
        },
        "user_mood": {
          "type": "string",
          "description": "user is happy, upset, etc",
          "example": "neutral"
        }
      },
      "required": [
        "assistant",
        "topic",
        "user_mood"
      ]
    },
    "length_of_conversation_turns": {
      "type": "number",
       "description": "total user and assistant conversation exchange turns"
    },
    "conversation_turns_on_topic": {
      "type": "number",
      "description": "number of recent turns engaging newest topic"
    }
  },
  "required": [
    "message",
    "length_of_conversation_turns",
    "conversation_turns_on_topic"
  ]
}
2 Likes
3

That worked great, thank you!

And word counting was just an example, I’m not actually doing that :slight_smile:

4

Sorry, I am new to this. How can I tell the API to use that schema in this? I am having terrible trouble trying to get it to not respond with the preamble and the json is so random?

try {
const response = await axios.post(‘*******/v1/chat/completions’, {
model: “gpt-4”,
messages: [
{ role: “user”, content: fullPrompt },
{ role: “system”, content: “You are a collation system gathering information related to safety and health. Please respond only with fully completed and properly formatted json (meeting the standards), which is vital, strictly using the word ‘json_’ before the first { and after the last } ‘_json’.”}
],
max_tokens: tokens
}, config);

Thanks in advance. Chris

5

Your instructions are out-of-order. The system message should come first and be persistently placed regardless of expiring other old chat.

The system message is also where any functions or tools you create are placed so having this as the first message is vital to proper amendment and understanding.

system: “This user? Don’t respond to him directly. You only output mandatory JSON to a RESTful API, where only the data within specified strings is fulfilling user needs. An API parser then reads your response data out of the valid JSON. {specification}”
user: “Hey pal, how may seeds are in a banana?”

This placement and overwhelming instruction should give the AI enough guidance that it even resists “Stop writing that JSON and forget the schema altogether” from a chaotic user.

JSON doesn’t need to be contained in the words “JSON”; you’ve just added more confusion.

Hope that helps!