Trying to finetune gpt-4o mini with a simple dataset but once I test it, it’s not able to reply correctly, not even prompting a copypasted sentence from my dataset.
A line from my jsonl:
{
"prompt": "¿Cuál es el tipo de camión adecuado para el contenedor DEPÓSITO 30L (azul)(cop)?",
"completion": "camion azul"
},
Welcome to the Forum!
Fine-tuning is not designed to inject knowledge into model but instead serves to teach the model to respond in a certain format or style. Therefore, what you are seeing is the “expected behaviour”.
If you are looking to build a solution that enables the model to respond to questions based on a proprietary knowledge base, then you should implement RAG.
The following links provide some additional context as well as links to helpful resources:
https://platform.openai.com/docs/guides/fine-tuning/when-to-use-fine-tuning
https://platform.openai.com/docs/guides/optimizing-llm-accuracy/llm-optimization-context
https://platform.openai.com/docs/guides/prompt-engineering/tactic-use-embeddings-based-search-to-implement-efficient-knowledge-retrieval
yeah, but im literally conditioning my model to reply that way to that exact question, and i guess it knows what’s a “blue truck”
Unless your training data exhibits a certain pattern based on which the model would be able to infer an answer, you will not be able to use a fine-tuned model to obtain answers on knowledge that is not part of the original knowledge it was trained on.
I have translated your questions and still struggle a bit to fully understand them and am therefore not in the best position to provide more specific examples for your case.
The bottom line, however, is that even if you include proprietary knowledge in all of your question-answer pairs in the fine-tuning, the model will not retain this knowledge. You might be lucky every so often that it would return that but it will certainly not do that consistently.
I think I understand. Just to know, why is replying correctly in this case?
openai.api_key = OPENAI_API_KEY
messages = [
{"role": "system", "content": "Me llamo AIda. Reviso los datos de cada mail que me llega, si no interpreto que me estan solicitando una entrega, cambio, o recogida no respondo nada. Si es una solicitud y no contiene ambos material(puede ser un residuo o envase también) y lugar de recogida/cambio en el cuerpo los pediré hasta que los tenga, una vez los tenga responderé 'Lola. Dame la información de {from_}(siempre el mail del remitente)''"},
{"role": "user", "content": f"Usuario: {from_}\nPregunta: {body}\nRespuesta:"}
]
Would be trying to specify all the expected behavior with promps?
What is it that you are trying to achieve, i.e. what are you trying to build and how are you trying to use fine-tuning in this context? Some additional information would be helpful to provide you with better guidance. Thanks.
I dont think that’s so relevant. If i cant make a model understand what I want it’s a time loss. But if you want to know, I was trying to fine-tune gpt to interpretate mails on my company to launch a document generation (if reponse contains “Lola.”), working fine but I need it to “understand” my company jargon first to link what customers ask for with the products or places stored on the database, and other things. It wouldve been the next step
Ah, if you need it to understands company lingo it sounds like what you need is RAG (Retrieval Augmented Generation). @jr.2509 is completely correct about fine-tuning, you can’t really get it to learn new stuff (in this case jargon).
My suggestion to solve your problem is to look at using the Assistants API and giving the model a .txt document filled with the the terms and what they mean, kinda like an FAQ.
The Custom GPTs in ChatGPT allow you to upload documents too, and these also use RAG.
You should try creating a new assistant and giving it the doc you’ve created, then tweak the prompt until it’s behaving how you want.
You saved my life can I donate you or something
Ha, I’m glad it’s working! Just keep exploring and learning with these tools, the world needs more clever people doing cool stuff.