I’m finishing up my thesis, which compares GPT’s performance on question-answering tasks, comparing the outputs from a fine-tuned model with those generated via grounding/context-injection (which was pioneered by @raymonddavey).
My professor is concerned that we can’t describe how GPT’s fine-tuning operation affects the base model. Does anyone have any information, preferably scholarly research that explains what is occurring?
I can’t tell you how OpenAI does their secret sauce, but I can give you a brief outline of low rank adaption the lora you hear so much about.
The basic requirement is, you would like to alter the main models weights with some new data set, doing that to the main dataset would take months and cost millions, not good, but there is an interesting fact about matrix math: given an n by n matrix A, you can find matrices U (n by m) and V (m by n) such that A is approximately equal to the product U*V where m is small.
While this is true in only some cases, in this use case it is, so you can take the very large n by n matrix of the GPT model and get the product of it and the MUCH smaller n by m “fine tune” and get a good approximation of a new model with updated weights.
Is that how GPT does it? No idea, but it’s how I would do it.
I have also been combing through the posts and tried many internet searches in an attempt to understand what is really happening under the hood when we fine-tune an openAI model using the fine-tune api. Something must get updated since we are setting the number of epochs and selecting a learning rate. Training and validation loss is calculated at every step.
When reading on how to fine-tune open source LLM, we indeed constantly bump into low-rank adaptation or LoRA. I like this picture from this this post
[Preformatted text](https://blog.devgenius.io/fine-tuning-large-language-models-on-azure-machine-learning-358338f4e66a)Preformatted text``
But the question remains, can someone actually confirm that is what is happening under the hood when fine-tuning an OpenAI model? Like many of us, I would really like to know.