I am using Structured Output and passing a JSON schema to the Chat Completion API. I am looking for the LLM to provide a “confidence score” for each object in an array returned in the JSON. I have defined this element this way:
gen.writeFieldName('confidenceScore');
gen.writeStartObject();
gen.writeStringField('type', 'number');
gen.writeStringField('description', 'Your confidence (0-100) that this load data is correct');
gen.writeEndObject();
I am getting way overconfident values back from the model. Sometimes “100” when I provide gibberish as the user input.
How can I craft a better description or definition in the JSON so I get a better confidence score?
Thanks for your help!!