I'm bad in prompting when I use json mode

"Hello, can someone help and explain to me why ‘justification’ and ‘wellnessvalue’ are always empty? And if anyone has a solution to fill ‘justification’ and ‘wellness value,’ please share it

{
  "response_format": {
    "type": "json_object"
  },
  "messages": [
    {
      "role": "system",
      "content": "You're going to calculate a well-being percentage per room and place it in the variable 'wellnessvalue.' Additionally, you'll write a justification for this percentage in the variable 'justification.' You MUST ALWAYS return your result in JSON format, and these two variables should never be empty or have '0' or 'null'."
    },
    {
      "role": "user",
      "content": "4 rooms: [name: Studio, humidity: 38.54, temperature: 26.43, pressure: 998.66], [name: Cockpit, humidity: 32.54, temperature: 27.43, pressure: 998.66], [name: Loft, humidity: 18.54, temperature: 16.43, pressure: 278.66], [name: Hall, humidity: 37.74, temperature: 16.43, pressure: 755.66]"
    }
  ],
  "functions": [
    {
      "name": "calculate_wellness_value",
      "description": "Calculates the wellness value and provides a justification.",
      "parameters": {
        "type": "object",
        "properties": {
          "rooms": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "Name": {
                  "type": "string",
                  "description": "The name of the room"
                },
                "temperature": {
                  "type": "number",
                  "description": "The room's temperature in degrees Celsius"
                },
                "wellnessvalue": {
                  "type": "string",
                  "description": "The well-being score calculated for this room from 0 to 100"
                },
                "justification": {
                  "type": "string",
                  "description": "The justification for the calculated well-being score in a sentence"
                }
              },
              "required": ["name", "temperature", "wellnessvalue", "justification"]
            }
          }
        },
        "required": ["rooms"]
      }
    }
  ],
  "function_call": "auto",
  "temperature": 0.7
}

If I understand correctly, in your example you’re kind of asking to calculate the wellness value before actually calling calculate_wellness_value.

What I’d recommend is actually change this to be a completion “task” (described in system message) instead of function calling.
Here’s what I’ve tried which I think kind of worked (didn’t look too much into the correctness of the response):

notice- no function call.

Here’s the system message I’ve used:

Calculate a well-being percentage of the provided rooms.
Additionally, you'll write a justification for this percentage in the variable 'justification.' 
Return a json object with the following parameters:
"roomsInfo": array of objects containing the room info, wellness value and justification for the wellness value. Every object should have the following schema:
"name": The name of the room
"temperature": The room's temperature in degrees Celsius
"wellnessvalue": The well-being score calculated for this room from 0 to 100
"justification": The justification for the calculated well-being score in a sentence

And here’s the response I got:

{
    "roomsInfo": [
        {
            "name": "Studio",
            "temperature": 26.43,
            "wellnessvalue": 85,
            "justification": "The Studio has a comfortable temperature and moderate humidity, contributing to a high well-being score."
        },
        {
            "name": "Cockpit",
            "temperature": 27.43,
            "wellnessvalue": 80,
            "justification": "The Cockpit has a slightly higher temperature but still within a comfortable range, leading to a good well-being score."
        },
        {
            "name": "Loft",
            "temperature": 16.43,
            "wellnessvalue": 60,
            "justification": "The Loft has a lower temperature, which may not be ideal for everyone, resulting in a moderate well-being score."
        },
        {
            "name": "Hall",
            "temperature": 16.43,
            "wellnessvalue": 55,
            "justification": "The Hall has a lower temperature and slightly higher humidity, which may lead to a lower well-being score for some individuals."
        }
    ]
}

** BTW the screenshot Promptotype which is super useful for development of these use cases (disclaimer: I created it)

2 Likes

Thank you so much for your response. I actually thought I wouldn’t need to fill out a task list since each variable has a definition. In the same time, I’ve updated my prompt and provided more precision on calculating the wellness value, emphasizing the need for the variables to always be filled after requesting the JSON.

I just have a concern with your method, it’s that the ‘required’ field is not present and maybe some var won’t be present

Happy to hear that (kind of) helped.

I think I understand your concerns with the schema- you’d prefer to use the “built-in” schema definition provided with function calling, however- iiuc function calling is not suitable for your use case. function calling’s schema is used for the arguments, while you’re trying to use it for the result (sorry inn advance if I’m missing anything).
Other than that, I think your concerns about the result’s schema are mostly solvable with the system prompt. There’s about of testing and fine tuning there, but it’s definitely solvable.

I’d recommend again to try out Promptotype which is very useful for developing and testing of this use case. It also supports function calling if you want to keep exploring that path.

1 Like

I was using Postman, to be honest, but I’m going to switch to your tool, which looks really cool. Thanks again!

1 Like

The biggest advantage over tools like postman is the ability to save expected json response (value or just schema) per query, along with creating a whole collections of queries to test together.
This is very useful since even small changes to the system message (or function) can sometimes have very significant effects (more than you’d expect).

in my experience, temperature > 0.5 causes formatting inconsistency even when function calling or using json mode.

2 Likes

Oh thank you but do you know why temperature > 0.5 is bad ?

No but I’d love to know if you would be so kind.

The models are tuned to produce specific outputs, high temperatures allows a larger range of probable next tokens to be taken into account, this allows the randomness to have a stronger effect, once one out of sequence token is generated it is likely that the sequence will continue to diverge. Lower temperatures reduce this risk.

2 Likes

Yes I understand how temp works but I do not know why it is “bad” like the previous commenter said.

If I need a higher temp for my output AND I also need json mode, I need two calls. One for the high temp output and a second call for json mode. I will use a temp of zero for the json mode call.

TL;DR:

JSON mode is not specified specifically for functions, nor would you want to write a function specifically for just receiving output from an AI.

Functions are for enhancing the AI abilities, by providing it external resources that can offer more insight or take actions to affect the outside world.

The AI is not trained to use a function as a way to write a response to the user.

JSON mode, only available with newest preview models, increases the probability of producing standards-compliant JSON output. JSON data format still needs to be specified fully to receive the type of output you expect, but specified in system message, not by function.