I don’t know the details, of how OpenAI is doing it, but to provide you with a quick educated guess: it’s likely a version LoRA fine-tuning ([2106.09685] LoRA: Low-Rank Adaptation of Large Language Models) where only a few parameters of the model are adapted to optimize for the new data.
But coming back to your question: YES, it does change (some) of the original parameters.