Open Assistants API with multi user chat

The tricky part is an assistant for a given conversation can only process a single request at a time so if you try to send two request simultaneously to an assistant the second one will fail. You have to poll (which sucks) and for the assistant to finish processing the current request so you need some way of blocking the other users until the current request finishes. The best way to do that would be to put a message queue between the users and the assistant. I couldn’t do that so I created a virtual mutex. It works

1 Like