Building a Q&A with multi workspace and users system like notion

We have a workspace mechanism, where there can be multiple users in a workspace and users can be into multiple workspace.
users can create their own documents using WYSIWYG editors. private or shared.
so a document has a permission who can access it.

We want to build a Q&A system using OpenAI assistant, threads and vector stores.
catch here is that when you create threads, user should be able to talk through only accessible documents.
We have thought of creating an assistant per user, but it can only have a single vector attached to it.
and users share data so creating unique vector for each assistant is a nightmare, considering that 100 user workspace would create 100 vectors with ~90% common data.
and maintaining 100 vectors for updating single document is another nightmare.

Is there any better approach?
Primary problems we want to solve is

  • Maintain Q&A history with threads for continuity of a conversation
  • Only talk with accessible documents.