Hello everyone,
I know that this topic have already been mentioned before, but I didn’t find the answer I was looking for.
As mentioned in the title, I aim to finetune a multi-label classification. Please note that a multi-label classification is not a multi-classification. Every prompt can belong to one or multiple classes. Let me give you an example of how I constructed the train.
{"prompt":"Barcelona sweatshirt club - black & gold","completion":" 11 a side football\n5 a side football"}
Here you can see that the description/prompt belongs to this two classes at the same time. Here is a second example where it can belong to three different classes:
{"prompt":"adult judo and aikido sports uniforms 150 White","completion":" aikido\njiu-jitsu\njudo"}
I have 20K lines like those on which I want to train my model. I have tried curie and babbage.
Questions:
- Is my data preparation correct?
- Why gpt3 suggested two different ways of postprocessing/preparing the data (changing suffixes, stopwords…) between curie and babbage model.
- How to limit him on not generating other classes other than what I gave him? I tried to put temperature to 0 but still continued to generate new words (less but still generating)
- How is it possible that babbage gave better results than curie for the same given data. (Curie was more generating new words and babbage was sticking more on the classes I gave him)
Thanks