Fine tuning models to generate JSON response

Prompting
, , ,
1

hey everyone! I am trying to fine tune a model where prompt will be string and completion will be a stringified JSON. I tried creating this model but the api reponse is always a string rather than a JSON. I am expecting it to return a strigified JSON with the structure similar to what i provided in completion?
What should I do to get the prompt and completion right and which model will be best for this?
P.S: The JSON i am passing in completion can get complicated with different use case.

2

The model has trouble with JSON.
I’ve found more success using Markdown where possible, or if not, yaml.
Markdown with a template as prompt data, and then asking the model to fill it in, works pretty well!
I can then parse the markdown and turn it back into JSON in my API caller.

1 Like
3

Thanks for the response. But in my case, i have JSON object with multiple levels.

key1 : {
 key2 : {
   key3 : [jajsnasas],
   key4 : ...
}

Something like this. Anyways I can fine tune my model which can return this deeply nested objects as well?

4

In almost every case I see where a fine-tuned model was created, it wasn’t necessary. Can you share the reason you chose this approach instead of the many other pathways to generate JSON?

5

Sorry I dont know any of the other ways. Can you share some material where I can find it?

6

Okay, no. There are many pathways to AI success, but no single pathway should be suggested without at least a general idea of the objective. Share your goal and I’ll be happy to provide some ideas.

7

In case it helps,

For several months I’ve been getting solid JSON responses back from the gpt-4 model by providing a format in the prompt and also having the system message say:

“All of your responses should be in UTF8, following the provided JSON format in each request, with no extra commentary.”

Yesterday, I started hitting the gpt-4-1106-preview model and suddenly about half the responses came back with triple backtick quote wrappers. I tried amending the system message:

“All of your responses should be in UTF8, following the provided JSON format in each request, with no extra commentary or quote wrappers.”

But it didn’t fix the problem.

Fortunately, with this model, they have added a parameter for requesting JSON output. You still absolutely need to have a system message like the one above, or it will generate tons of whitespace until context is exhausted. But now you can also specify the response format on the API request like this:

    const request = {
        model: "gpt-4-1106-preview",
        response_format: {"type": "json_object"},
        messages: [
            {"role": "system", "content": systemMessage},
            {"role": "user", "content": prompt}],
    };

Now it’s back to its old self, reliably returning consumable JSON responses.