Caastrophic forgetting in OpenAI models

Let’s say if i have a finetuned GPT-4 model.

Now, after some time, I want to further add knowledge or more data on that finetuned model, would that finetuned modell face catastrophic forgetting problem?
If yes, is there way to overcome ?

I’m not sure that I understand your question, but fine tuning is not meant to add knowledge to a model, for that you’d be better off using RAG (which the assistants API uses).

2 Likes