I am using gpt-realtime API and it works for 2-3 times before it starts returning 403 Unauthorized error. Note that I am using ephermal key as the Bearer for the realtime endpoint that I get from the session endpoint.
One strange thing is that if I create a new API KEY and use it in the code, it again works for 2-3 times and then starts returning 403 Unauthorized.
I am having difficulty understanding this weird behavior. Note that I have no limits set and have enough balance in my account.
What must be causing this issue?