I have the feeling that if I upload additional knowledge about files via retrieval, the assistant only uses these files to answer the requests, but no longer the “basic knowledge” of the model.
No, the retrieval system adds additional, not replacement, context to the system.
My experience has been, you have to tell the assistant explicitly to reference the files.
Depends on how the system prompt is engineered.
Making an explicit statement like “Rely on files provided for x,y,z” does work.
Improve the same by including some conversational examples within the system prompt.
If you are keen on leveraging model ability to predict with native file data, highlight a clear order. Example:
“Always get x,y,z from the file first. If and when information is not available predict the best outcome.”