One or many assistants for SAAS?

API
1

Hello, I want to build a SAAS like chatbase where users can create their own AI agents and provide instructions, functions, and files for it to use. They will be able to embed the chat widget on their websites, or connect it to Messenger or Instagram.

Before I begin the project, I would like to get a second opinion on the structure:

I intend to build the backend in either FastAPI or NodeJS, and I am primarily thinking about two structures for using OpenAI’s assistants. Either I create one assistant per bot, so when a user in our SAAS creates an agent and adds a system prompt etc, that is immediately created as an assistant in OpenAI. The other structure I’m thinking about is to create one assistant for all users and attach the user’s vector storage, instructions, and functions on thread creation. The vector storage would in that case already be created for each user’s bot so it can be attached easily.

What would be the best approach? What is the “standard” way to do it?

I don’t want to have a long latency when a customer initiates a conversation, and I would prefer a solution that enables me to connect the agents to a phone service for taking calls in the future, so low latency is important.

Thanks in advance!

2

Don’t use Assistants, they are deprecated. Similar functionality is/will be provided by the Responses API.

However, if you want “snappy” performance, I would strongly suggest using Completions.

That will also give you more flexibility on what services you use to store vectors, files, etc., otherwise you will basically be a thinner wrapper to OpenAI and all the implications of that (including less control over costs)

Wrapping the Reponses API will be easier, but it will be harder to differentiate yourself from the hundreds of other businesses doing exactly the same thing.

And whilst I like to promote OpenAI, it’s worth noting that going down that route will also tie yourself more into their ecosystem.

ie, stick with Completions.