Fint-tuned model long responses and very slow responses

API
1

Hi all,

I am fooling around with GPT and trying to fine-tune my own model, which I did with 3700 examples.

Now when I type a question I always get a response that is bad, not relevant and very very long, so long it cut offs.

I have tried both using prompt and completion and just completion, no difference.

Any ideas why the model acts like this? Can it be that I didn’t use cut-offs (e.g \n etc)?

1 Like
2

Welcome to the forums.

It could be any number of things. 3700 examples is a lot, and should have some effect.

Can you show an example of the dataset and formatting you used?

What temperature and other settings are you using for completion?

3

My first fine tuning resulted in the same bad output. The key for a better outcome in the 2nd fine-tuning was the structure of the samples.

I did something of this sort:

Prompt: INPUT1: <some input 1>\n\nINPUT2: <some input 2>… \n\n###\n\n , Completion:

I built my jsonl with this, fine-tuned the model, and then I would ask questions with the same structure of the prompt.

Go to playground
Selected your fine-tuned model
Enter question like:

INPUT 1: <some new input 1>
INPUT 2: <some new input 2>
…

Click Submit.

Bingo!

INPUT x is anything you want/need and it depends on your specific task or case.

Thoughts?

Juan

4

Thanks!

This is a random example from the file:

{“prompt”: “”, “completion”: “För ett aktiebolag som följer K2­reglerna är det Bokföringsnämndens allmänna råd BFNAR 2016:10 som bestämmer vad som är god redovisningssed.”}
{“prompt”: “”, “completion”: “Årsredovisning enligt K2 När du väljer att upprätta en årsredovisning enligt K2­reglerna måste du följa samtliga regler, dvs de regler som vi beskriver i denna bok.”}
{“prompt”: “”, “completion”: “Du kan alltså inte välja att tillämpa bestämmelserna i K2­reglerna för inven tarier och sedan välja att värdera varulagret enligt K3­ reglerna.”}
{“prompt”: “”, “completion”: “Reglerna ska alltså ses som en helhet och tillämpas till­ sammans.”}

I have also tried using a prompt, but not difference.

I have tried difference settings, example:

  temperature: 0.2,
  maxTokens: 1500,
  topP: 0.9,
5

Thanks for this. I might try adjusting my training data a bit more just to see the outcome.

Will trim it down a bit, cost a bit to run these massive training data :slight_smile:

6

Is your goal to be able to ask questions about the data to the model? or is it to have the model answer in the style of the data?

These are 2 very different goals as far as the training goes.

If your goal is to have the ability to ask any question about the data and have answers that are mainly ‘true’, meaning that the answer is knowledge found in the data, then fine tuning may not be the way. You’ll want to try to use embeddings.

If your goal is to have the model to completion in the style and focused on your data, but not necessarily respond with ‘truth’ to your data, then fine-tuning may be your path.

Please share some more about your use case.

Thanks,

Juan

2 Likes
7

I was having trouble with long responses as well and limiting the number of keywords to return didn’t seem to be helping consistently. I added text to my message to say “in 100 words or less” and then it seemed to consistently give me the type or short replies I was looking for.

8

The use case is that users ask specific question about certain topics, that the fine tuned data covers, the answer should related to this. It is like a support bot regarding this specific area.

9

Then you want to look at embeddings, not fine-tuning.

Check it out and if you have any questions I’ll be happy to assist as much as possible.

2 Likes
10

I tried using embeddings via Chroma, got a file looking like this (based on the same file I used for fine-tuning my model):

[[-0.19246907532215118, 0.9037320017814636, -0.24110646545886993, -0.8702456951141357, 0.4042340815067291, 0.3157321810722351, 0.7959880828857422, -0.5493348240852356, 0.1047724187374115, -0.277881920337677, -0.12633928656578064, -0.3377610445022583, -0.308800607919693, -0.3110261857509613, -0.2797357439994812, 0.4647640287876129, -0.039902687072753906, 0.3989901542663574, 0.5787240862846375, -0.06768105179071426, 0.0003971833793912083, -0.353933185338974, 0.008713806979358196, -0.0442199744284153, 0.4402839243412018, -0.8958728313446045, -0.19572432339191437, 0.5904616117477417, 0.4807640016078949, 0.7421479821205139, 0.5070416331291199, -0.012438076548278332, 0.013397383503615856, -0.35009825229644775, -0.43321382999420166, 0.4547211527824402, -0.9580157399177551, -0.4105294942855835, 0.3650394380092621, 0.6870201230049133, -0.2832156717777252, -0.678849995136261, 0.4197622239589691, -1.273140788078308, 0.7840603590011597, 0.2522733211517334, -0.14799906313419342, 0.0007855601725168526, -0.167530819773674, 0.747226893901825, 0.01461015548557043, -0.9564000368118286, 0.07305273413658142, 0.020260609686374664, 0.060814667493104935, 0.10540225356817245, -0.04268593713641167, 0.11765870451927185, 0.4274815022945404, -0.5583266615867615, -0.12256475538015366, -0.8920667767524719, 0.6904195547103882, 0.14782680571079254, -0.4377835988998413, -0.6422802805900574, -0.06376843899488449, -0.0862789899110794, 0.31636959314346313, -0.2956666052341461, -0.4890243113040924, 0.9153268933296204, 0.05327567830681801, -0.7175175547599792, 1.0538102388381958, -0.6215469837188721, -0.09197873622179031, 0.15299293398857117, -0.42608270049095154, -0.01701119728386402, 0.9107618927955627, -0.6074125170707703, 0.024996565654873848, 0.06947872042655945, 0.26050087809562683, -0.19140584766864777, 0.6780357360839844, -0.705043613910675, -0.16641291975975037, 0.1970238983631134, 0.19255231320858002, -0.8616503477096558, -0.11658409982919693, -0.5093193650245667, 0.20860114693641663, …]

I havent figured out how to integrate all this with my GPT model yet.

I tried using the OpenAI embeddings solution, but it didnt’t seem to work (the file didn’t look correct).