How to assign weights during chat based fine tuning?

API
,
1

I am trying to create a chatbot which converts user’s natuaral language query into an SQL query. Now my system prompt is pretty big, about 1884 tokens, but it gets the Job done pretty well in most of the cases.
I am using gpt-4o for this task but I am trying to switch to a Fine Tuned version of gpt-4o mini to save cost and make it work on complex sql queries.

I read the fine tuning documentation which stated this:

Multi-turn chat examples:

Examples in the chat format can have multiple messages with the assistant role. The default behavior during fine-tuning is to train on all assistant messages within a single example. To skip fine-tuning on specific assistant messages, a weight key can be added disable fine-tuning on that message, allowing you to control which assistant messages are learned. The allowed values for weight are currently 0 or 1. Some examples using weight for the chat format are below.

{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris", "weight": 0}, {"role": "user", "content": "Can you be more sarcastic?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already.", "weight": 1}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "William Shakespeare", "weight": 0}, {"role": "user", "content": "Can you be more sarcastic?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?", "weight": 1}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "384,400 kilometers", "weight": 0}, {"role": "user", "content": "Can you be more sarcastic?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters.", "weight": 1}]}

I still fail to understand when I should be assigning weight value as 0?

Here is an example scenario:

SYSTEM: {My system prompt which has tables and 11 few shots defined}

HUMAN_MESSAGE_1: "Who are the top exporters of coffee? "

AI_MESSAGE_1: " The top exporters of coffee are Tarla Beans, Magic Plaza, Khalid Kashmiri and Mohammed sumbhul. If you want to know more let me know!"

HUMAN_MESSAGE_2: “What is the total shipment cost done by the first and third exporter?”

AI_MESSAGE_2: “SELECT exporter_name, sum(total_value) as val from trade_table where exporter_name in (‘Tarla Beans’, ‘Khalid Kashmiri’) GROUP BY exporter_name ORDER BY val”

So as you can see, the sql query does depend on AI_MESSAGE_1s output. So should AI_MESSAGE_1 get a weight of 1 or 0?

1 Like