Training a model for specific output format rather than content

Prompting
,
1

I want to train a model so that it output specific JSON values in the correct format relative to user inputs. I’ve tried including instructions as the assistant output to my messages, but its clear now that that isn’t how fine tuning is designed to work. The reply from the assistant is simply my instructions formatted as JSON, it did no replace any of the instructions with actual content.

For example:

{"role": "system","content": "Return me a random Colour & Object in JSON format"},{"role": "user","content": "Tell me a random colour and object"},{"role": "assistant","content": "\"JSON\"{\"value1\":{\"content\":\"tell me a random colour\"},\"value2\":{\"content\":\"tell me a random object\"}}"}

Returns this:

"JSON"{
"value1":{"content":"tell me a random colour"},
"value2":{"content":"tell me a random object"}
}

I want to to return a random item without me having to define it:

"JSON"{
"value1":{"content":"Blue"},
"value2":{"content":"Car"}
}

Most of the examples ive seen are based on content output as text rather than any specific format.

Is there a better way to train models to focus on learning only the appropriate format of outputs and then to creating the content itself, rather than it following any specific content that i include within the JSON format?

2

Hello @Gazza ,
try to use new models gpt-4-1106-preview or gpt-3.5-turbo-1106. They accept the response_format parameter:

"response_format": {"type": "json_object"}

Your system and user prompts are ok.

Here is the complete documentation of the API: https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format

3

Are these models available for fine tuning?

For complex scenarios I find that they often fail to follow instructions regarding JSON format.

4

theres a parameter called ‘functions’ which is basically function calling.

It’s putting guard rails to always return JSON object and structure that you want.

i have pretty complex fuction and not once have i gotten output outside json format. something it knows to haluccinate additional values, but that only happens when you over-burden model, you have to find sweet spot.

Here’s an example:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use. Infer this from the users location.",
                    },
                },
                "required": ["location", "format"],
            },
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_n_day_weather_forecast",
            "description": "Get an N-day weather forecast",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use. Infer this from the users location.",
                    },
                    "num_days": {
                        "type": "integer",
                        "description": "The number of days to forecast",
                    }
                },
                "required": ["location", "format", "num_days"]
            },
        }
    },
]
1 Like
5

Currently of the two new models only gpt-3.5-turbo-1106 can be fine-tuned (more here: https://platform.openai.com/docs/guides/fine-tuning/what-models-can-be-fine-tuned).
But I think you can first try the new parameter response_format. I found it very effective. Just remember to instruct the model to produce JSON, via a system or user message.

6

Is it possible for the AI to choose the best suited JSON format itself, if provided multiple functions with varying JSON formats?