I can see that if the WS session on the realtime API dies, as long as you’ve stored the session interactions, you can populate them again in a new session. My question is, does populating them again mean these get charged as new input tokens?
If yes, then this gets very expensive very quickly.
there is caching now, but I haven’t extensively test it yet, didn’t get it to work last time I tried.
I guess with anything regarding generative pre-trained models APIs can get expensive. If you loop gpt-3 and leave it running it’ll get expensive, bonus points if you do it with the o1-preview model.
now, to your point, every “conversation” you have is like a string, right? So think of string that grows as the conversation goes. When the conversation gets long, the amount of tokens you are using grows as well. So populating them again will cost the same as if you it was already populated.
That’s what I think, last time I tried using the realtime api it was still giving the 403 error, by any chance, did you get it to work?
on the openai-realtime-console you should be getting Could not connect to "wss://api.openai.com/v1/realtime" and on the simple api request you should be getting WebSocket Error: Error: Unexpected server response: 403