Unable to get reliable response from Fine Tuned model

Hi All,

I have a very specific need where I want to train an AI model to read some delimited data and identify some data points and return those data points. I have a specific list of labels/headers I want it to search and return the data points in a delimited format of header and values, if found. The major challenge here is that the format of the delimited data is not fixed. I have tried to create a diverse set of examples and included it in my training JSONL file. I have around 28 datasets in training file and 4 datasets in validation file. The numbers seem fine

Now the problem is that the fine tuned model keeps juggling data. Like, for example: I have a column for NAICS code which should only get a unique numerical code: my model returns a text from 2 columns ahead and moves all the other values along with it. It is able to pick the required data but keeps moving it to wrong columns. Sometimes it works without a hitch and sometimes it wont give me correct position. I have set temperature to 0 and top P to 0.5
I have also tried a lot of different combinations of training parameters epochs, batch size and LR multiplier.

Could anyone please point me to the right direction and help me understand what I am doing wrong?

Thank you.