What I am doing wrong? fine tune

Hi! I am starting in this interesting world. The idea of my app is to use a model to extract certain information of a text (pdf’s). The main information is extracted ok but some fields sometimes are wrong. I tried to tune a model though a jsonl file as i.e.:

{“messages”: [{“role”: “system”, “content”: “This is a bot that identifies specific information in texts. Requested to extract data from the text”}, {“role”: “user”, " content": “if the text mentions or contains the url blablabla(dot)com, which agency does this correspond to?”}, {“role”: “assistant”, “content”: “The agency mentioned is always ‘cocacola’”}] }

and then I use the result model to extract for specific keywords from a pdf’s but the agency is still always wrong, don’t know what I am doing wrong. Any help is apreciated. Thanks! :slight_smile:

1 Like

Hi @jviure - welcome to the community!

Hopefully we can help you out here. I’d like to give it a start, however have a few additional questions based on the example you provided.

  1. System message: It sounds as if your second sentence “Requested to extract data from the text” has the same meaning as the first one. Is this correct? If not, could you please explain the difference in meaning between the two sentences.

  2. User message/content: In your application of the fine-tuned model, what would normally be the user message? Is it a user question or is it input from a PDF?

  3. Assistant message/content: Are you looking to just extract a specific information such as the word “cocacola” or does it need to be a full sentence?

1 Like

Thanks for your answer. Well, I am developer, not used to work with IA, I am trying to train an openia model (gpt-3.5-0125), if the user input contains some reference to the url blablabla(dot).com is sure that the agency correspond’s to ‘cocacola’ always (so, in the json answer I ask it, return {… agency: ‘cocacola’,…} . But don’t know if it is possible to do that. :slight_smile:

What you are trying to do is technically possible. I am still not sure if I understand your example well enough to assist though. That said, another member of the Forum yesterday made a detailed post about the approach to entity/information extraction, which you may find helpful.