How would i scale the OpenAI Assistants API for multiple users

API
, ,
1

Hello,

I have a multi-modal agent swarm system setup that I would like to expose as a custom backend for my SaaS project. I am a little lost as to how i would handle multiple users without forcing my users to create an OpenAI Account and register their own API Key. I would like to prevent that for multiple reasons, 1. it confuses my pricing model - i would like to handle per-user api calls in my pricing model without requiring the user to be techical and pay me and openai separately. 2. I would like to handle all API requests via a single API key in my backend so that i have metrics and statics as well as scalablity within my API Gateway.

Right now, i have my Assistants working using threads and function calling - i would just like a little guidance as to how best to scale this to prod.

Settings up a proxy api would be ideal, however i really don’t want to write all of that myself.

Should i use my one api key as is and simply store the thread and message id’s in a relational data base relative to the user id? Is that enough?

2

Theoritcally yes it is enough, but in practice it may be different. I have not looked very hard, but I have not seen an assistants based production level application yet. I know for a fact having multiple users below 15 on one API key is feasible, but tracking metrics need to be done on your end as OpenAI’s API does not give enough breakdown to track at a user level.