Assistant api references information from uploaded files but not associated to the assistant

I have uploaded 5 documents under the Files area (doc1…5) using the platform openai site.
I then uploaded doc6, doc7 via files api call and associated it to my assistant gpt (purpose=‘assistant’).
However now if I query the assistant with information that is contained within doc4 (the first set of files uploaded and not associated to any assistant), the assistant is able to answer the query and present that information.
I would like the assistant to only query files that are part of a particular assistant.

Files Create API only tells that the purpose is ‘Assistant’ and hence ANY assistant is able to access that file! That is cross-contamination of information across assistant and IMHO a bug!
OR is there another way to instruct Assistant to only reference files that were uploaded to that particular assistant?

My use case was where user can upload files via a front-end application and query information from that document only. So this way I can create multiple assistants for various end use case [hotel information assistant vs. pharmacy information assistant] and not have hotel assistant provide me information about pharmacy in case the user asks that question.

When the assistant is created, you provide a list of file.ids , those should be the only files the assistant has access to. Have you seen an assistant knowledge retrieve files it was not assigned at creation?