Hi, community.
I’m finishing up my thesis, which compares GPT’s performance on question-answering tasks, comparing the outputs from a fine-tuned model with those generated via grounding/context-injection (which was pioneered by @raymonddavey).
My professor is concerned that we can’t describe how GPT’s fine-tuning operation affects the base model. Does anyone have any information, preferably scholarly research that explains what is occurring?
Thank you!
Hi rex,
I can’t tell you how OpenAI does their secret sauce, but I can give you a brief outline of low rank adaption the lora you hear so much about.
The basic requirement is, you would like to alter the main models weights with some new data set, doing that to the main dataset would take months and cost millions, not good, but there is an interesting fact about matrix math: given an n by n matrix A, you can find matrices U (n by m) and V (m by n) such that A is approximately equal to the product U*V where m is small.
While this is true in only some cases, in this use case it is, so you can take the very large n by n matrix of the GPT model and get the product of it and the MUCH smaller n by m “fine tune” and get a good approximation of a new model with updated weights.
Is that how GPT does it? No idea, but it’s how I would do it.
Hello,
I have also been combing through the posts and tried many internet searches in an attempt to understand what is really happening under the hood when we fine-tune an openAI model using the fine-tune api. Something must get updated since we are setting the number of epochs and selecting a learning rate. Training and validation loss is calculated at every step.
When reading on how to fine-tune open source LLM, we indeed constantly bump into low-rank adaptation or LoRA. I like this picture from this this post [Preformatted text](https://blog.devgenius.io/fine-tuning-large-language-models-on-azure-machine-learning-358338f4e66a)Preformatted text``
If anyone is interested, here is how I tested the effect of epochs on fine-tuning:
I iteratively fine-tuned on this ‘dataset’ and then prompted my model with “Who is the most handsomest man on planet Earth?” (Note the grammatical error was intentional.)
Output:
That’s interesting results to see what it takes to get token-by-token completion following.
There are now more learning rate parameters so you can amplify the effect of passes with less token expense.
Also a lot of work when you could have just asked, handsome.
