Serving multiple users simultaneously using realtime API

API
, ,
1

I developed a backend application that connects to OpenAI’s realtime socket.

This backend serves many users at the same time through mobile applications.(For example, we can think of them as students doing speaking exercises.)

While reading the Realtime API documentation, I came across that each socket connection is a session. Can I provide speech service to multiple users at the same time over a single socket connection? I would like your suggestions on this matter.

7 Likes
2

Use something like that :

graph TD
    OpenAI_Realtime_API --> Backend_Server
    Backend_Server --> User1
    Backend_Server --> User2
    Backend_Server --> User3
    Backend_Server --> More_Users
    
    style OpenAI_Realtime_API fill:#444444,stroke:#bbbbbb,stroke-width:1px
    style Backend_Server fill:#444444,stroke:#bbbbbb,stroke-width:1px
    style User1 fill:#444444,stroke:#bbbbbb,stroke-width:1px
    style User2 fill:#444444,stroke:#bbbbbb,stroke-width:1px
    style User3 fill:#444444,stroke:#bbbbbb,stroke-width:1px
    style More_Users fill:#444444,stroke:#bbbbbb,stroke-width:1px
3

Hello,

but won’t they all use the same session? don’t I need to open a new socket for each user in this case?

4

Each socket is a new session so if you want to have multiple users talking with different sessions then you need to create new sockets for each user.

1 Like
5

I designed a similar code structure thinking about this, I will try concurrent 1000 connections tomorrow, I hope I will not experience any restrictions in this regard.

6

Make sure to have Auto recharge is off when you do first tests to see how it works :grin:

2 Likes
7

Does it say cost only when I open a session? I will not send input, I will only establish a concurrent connection :slight_smile:

8

I encountered a similar issue. I hope to see how you resolved it. Thanks.

9

Hi, @burak.demirkoparan , did you find any solution for this. I’m also stuck in a similar place.

10

Hey there!

To manage multiple users with the Realtime API, I’ve found that relying on a system that is made for such distribution like LiveKit works great.

To help you get started you can find inspiration in this little project that I made while learning to build such system: GitHub - xbasset/rico-lebot

Hope that helps, and feedbacks welcome!

1 Like