Fine Tuning multi label classification dataset

API
1

Hello all,

I would want to fine-tune gpt-3 using a multi-label classification dataset i.e Few-shot learning. There are no examples or any relevant info given in the docs.

Any one had tried something with multi-label dataset?

Thanks,
Cassin

2

Hi @cassinthangam4996 did you manage to find a solution?? I am trying to find my way through this problem of multi-label fine tuning. thank you

3

I think I have found a solution to the multi-label classification problem. It appears to be working well so far. You can get up to 5 classes this way.

  1. Train the model with a single-token for each class if possible. This is not strictly necessary, though. I have found that if a multi-token label, say two words, provides more semantic clarity, it seems to work well with fewer false positives. I have not tried using numeric class labels.
  2. When sending prompts to your find-tuned model, add logit_bias to limit output to just the tokens that represent your classes. This means sending the token IDs for each of your class labels in the logit_bias section.
  3. Set logprobs = 5, max_tokens = 1
  4. Submit your prompt. Extract the top_logProbs from the output.

I am finding that in a lot of cases, anything > -4.0 is relevant, but I am still experimenting. Some classes are working better than others, of course.