How to Design the Final Output into a JSONL File？

API
1

I need to design a prompt that requires the AI(GPT-4o) to:

  1. Analyze the problem in detail and solve it step by step.
  2. Return only the final result in JSON format.

The challenge I face is: How to ensure that the AI conducts a thorough analysis while only returning a concise JSON-formatted result?

How can we implement the following two options: A: The API returns the complete analysis process and the final result in JSON format. B: The API returns only the final result in JSON format like

{
  "problem_uuid": "66f4f3527cab3d2c82c3be62",
  "student_id": "aa11",
  "comparison_result": {
    "is_correct": true,
    "correctness_percentage" ,
    "summary": " 。"
  }
}

Can someone help this ？ thank you

2

This requires 2 requests (I mean of course the model has to tell you about the reasoning else it did not happen - and I mean you also want to know if it happened).

  1. send the prompt for the analysis
  2. receive the answer
  3. add the messages for a second prompt where you do this:

Create a json schema and give it with the prompt.

This one is probably overkill af:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "problem_uuid": {
      "type": "string",
      "pattern": "^[a-f0-9]{32}$"
    },
    "student_id": {
      "type": "string",
      "pattern": "^[a-z0-9]+$"
    },
    "comparison_result": {
      "type": "object",
      "properties": {
        "is_correct": {
          "type": "boolean"
        },
        "correctness_percentage": {
          "type": "number",
          "minimum": 0,
          "maximum": 100
        },
        "summary": {
          "type": "string",
          "minLength": 1
        }
      },
      "required": ["is_correct", "correctness_percentage", "summary"]
    }
  },
  "required": ["problem_uuid", "student_id", "comparison_result"]
}

Also add as last sentence to the prompt:

Start the output with { and end the output with }

And I would also add some rule based evaluation e.g. with a regular expression get all problem_uuid from the document(s) and then count that and compare it to the output of the LLM.

Here is some pseudo code:

messages = []


messages = [{'role': 'system', 'message': 'you are a helpful intern - find the problem uuid and the student number and stuff and show me your reasoning in chain of thought style'}
{'role': 'user', 'message': 'here is some data to analyse: {some data}'}]


answer = model.chat($messages)


messages = [{'role': 'system', 'message': 'you are a helpful intern - find the problem uuid and the student number and stuff and show me your reasoning in chain of thought style'}
{'role': 'user', 'message': 'here is some data to analyse: {some data}'}
{'role': 'assistent', 'message': '{answer}' }
{'role': 'user', 'message' : 'follow up with the json schema and the last sentence where we ask for response start with { and end with }'}
]


json = model.chat($messages)
1 Like
3

thank you again .
you help me a lots of at the projcet

1 Like