Fine-tuning a Language Model to Generate dinamically specific JSON Structure without Prompting

How can I fine-tune a language model to generate a specific JSON structure without explicitly providing the structure in the prompt? For example, I want to generate an engaging text content for a popup widget in a JSON object that showcases our tech gadgets collection, with the following structure:

{
"type": "Text",
"props": {
    "elm": "#text",
    "arr_html": [
          {
          "html": "<h1>Embrace the Future with Cutting-Edge Tech!</h1>",
          "klass": "text-center"
          },
          {
          "html": "Discover the latest gadgets that will revolutionize your digital lifestyle.",
          "klass": ""
          }
       ]
    }
}

is it possible?

No, you must provide the raw format and at least one example filled in as you wrote above, but sometimes it may not work (Based on GPT-3.5). So sometimes, you may need to reprompt it and ask it to follow another strict example. Additionally, please define the datatype in your raw format. For example: “type”: “string”.

1 Like

An alternate approach, depending on the parts you actually need the model to generate, is to merge the response into an existing dictionary/JSON object.

e.g. maybe you prompt such that the model output is:

<header>
Embrace the Future with Cutting-Edge Tech!
</header>

<text>
Discover the latest gadgets that will revolutionize your digital lifestyle.
</text>

And then you parse that output and merge it into the JSON object as needed. In other words, if you don’t actually need the model the dynamically generate JSON, and you have a fixed JSON object you’re trying to populate, you might be better off just asking for the text you need from the model and programmatically populating the JSON object.

4 Likes

Thank you so much for your response and suggestions! I really appreciate your help. I’m currently trying to make the model generate the JSON dynamically based on the prompt. If it turns out to be too difficult or not possible, I’ll consider alternative approaches.

@dliden has pointed in the right direction, you can generate individual elements of the JSON with the model that fits your requirements and budget, ideally gpt-3.5-turbo and then simply create the JSON object on your end by passing the generations.

  1. It’s save you token costs.
  2. It can be done out of the box without fine-tuning

Alternatively you can few shot learning to generate the JSON, if you don’t mind the costs.

Just curious: why is it important for the AI to generate the JSON object?
It can be done easily programmatically.

Also, you can use ChatGPT to write the code that is needed to generate the JSON object programmatically.

In that case, I think the answer will depend a lot on the specifics of what you’re trying to accomplish. Sometimes it’s sufficient to say “return a JSON object with keys for x/y/z” though I suspect it would struggle somewhat with the nested structure from your example.

I also worked fairly extensively on a project that needed JSON output (you can read about it here if interested, though I’m not sure how directly relevant it will be). I ultimately found that relying on gpt-3.5-turbo to generate the needed JSON was not quite reliable enough for my uses. Populating a JSON object with the text generated by the model worked better. gpt-4 was considerably more reliable than gpt-3.5-turbo as well, if that’s an option for you.

1 Like

The entire issue, as I mentioned, is the reliability and stability of any requested form of output, whether it’s JSON or the use of delimiters or tags as suggested above. GPT-3.5 may still ridiculously disobey the requested format example at times, and any form of post-processing will fail. Therefore, always reconsider the recursive reprompt until the output generated by GPT can be understood by your JSON converter. (Specify your limit $$$ for this recursive reprompt)

1 Like

I have heard that gpt is better at consistently producing correct yaml than json. Fewer tokens as well. If that is true, should be pretty straightforward to convert the yaml to json.

Hi,

How is recursive reprompt achieved?
Does she have to validate the resultant JSON against a schema then send another .chatCompletion({}) ?

Hi

Did you guys have a situation where gpt [3/4] was returning the SAME response all the time. Even when the user prompt was varying?

Ive implemented something similar to what you detailed in your medium article, but the LLM is responding with a somewhat CACHED response on the JSON payload.

I have not encountered this—I’d carefully check your code if different prompts appear to be eliciting the same responses.

What you meant is, for every undesired response that is not covered in your JSON converter, add a new request to the USER + ASSISTANT loop, ensuring to provide the entire history of USER + ASSISTANT that you have used up to that point. You can use a simple prompt like “Your response was not satisfactory. Please redo it strictly following this model X.” If there is another error, continue repeating the same request “Your response was not satisfactory. Please redo it strictly following this model X.” At some point, the model will comply, and the result can be utilized by your converter. I hope this helps, at least it works very well for me.

Do not forget to define a limit to this loop, I set to 3 (Never got 3 yet)