Realtime API [kotlin?] WebRTC client stopped working in past < 36 hours?

API
1

I have an Android Kotlin Compose project that is a PushToTalk app that uses OpenAI Realtime API over WebRTC and has been working fine for months:
https://github.com/swooby/AlfredAI/

It was last known working less than 36 hours ago 2025/03/29 ~2am PST.

In the past ~36 hours it has stopped working.
It just hangs for ~10-12 seconds while negotiating SDP and then onIceConnectionChange(FAILED).

The main failing code is around:

The log (redacted and I attempted to shorten some):

2025-03-30 11:37:03.052  7558-7588  MobileViewModel         D  jobConnect: ephemeralApiKey="ek_{REDACTED}e41"
2025-03-30 11:37:03.052  7558-7588  MobileViewModel         D  -jobConnect()
2025-03-30 11:37:03.053  7558-7638  PeerConnection          D  onSignalingChange(HAVE_LOCAL_OFFER)
2025-03-30 11:37:03.054  7558-7638  PeerConnection          D  localDescription setSuccess()
2025-03-30 11:37:03.056  7558-7588  okhttp.OkHttpClient     I  --> POST https://api.openai.com/v1/realtime?model=gpt-4o-mini-realtime-preview
2025-03-30 11:37:03.057  7558-7588  okhttp.OkHttpClient     I  Content-Length: 1511
2025-03-30 11:37:03.059  7558-7588  okhttp.OkHttpClient     I  Content-Type: application/sdp
2025-03-30 11:37:03.059  7558-7588  okhttp.OkHttpClient     I  Accept: text/plain
2025-03-30 11:37:03.059  7558-7588  okhttp.OkHttpClient     I  Authorization: Bearer ek_...
...
2025-03-30 11:37:03.060  7558-7588  okhttp.OkHttpClient     I  a=max-message-size:262144
2025-03-30 11:37:03.060  7558-7588  okhttp.OkHttpClient     I  --> END POST (1511-byte body)
2025-03-30 11:37:03.066  7558-7636  org.webrtc.Logging      I  NetworkMonitor: Start monitoring with native observer -5476376645122786960 fieldTrialsString:
2025-03-30 11:37:03.071  7558-7636  org.webrtc.Logging      W  NetworkMonitorAutoDetect: Unable to obtain permission to request a cellular network.
2025-03-30 11:37:03.075  7558-7642  org.webrtc.Logging      I  NetworkMonitorAutoDetect: Network handle: 437197393933 becomes available: 101
2025-03-30 11:37:03.077  7558-7638  PeerConnection          D  onIceGatheringChange(GATHERING)
2025-03-30 11:37:03.082  7558-7642  org.webrtc.Logging      I  NetworkMonitorAutoDetect: handle: 437197393933 capabilities changed: ...
2025-03-30 11:37:03.084  7558-7642  org.webrtc.Logging      I  NetworkMonitorAutoDetect: handle: 437197393933 link properties changed
2025-03-30 11:37:03.085  7558-7638  PeerConnection          D  onIceCandidate(candidate="0:0:candidate:2383047984 1 udp 2122260223 10.0.2.16 52040 typ host generation 0 ufrag oZDr network-id 5 network-cost 10::UNKNOWN")
...
2025-03-30 11:37:03.087  7558-7638  PeerConnection          D  onIceCandidate(candidate="1:1:candidate:606096058 1 udp 2122005759 ::1 56020 typ host generation 0 ufrag oZDr network-id 4::UNKNOWN")
2025-03-30 11:37:03.153  7558-7563  swooby.alfredai         I  Compiler allocated 5250KB to compile void android.view.ViewRootImpl.performTraversals()
2025-03-30 11:37:03.186  7558-7638  PeerConnection          D  onIceCandidate(candidate="0:0:candidate:3387745661 1 tcp 1518149375 10.0.2.15 9 typ host tcptype active generation 0 ufrag oZDr network-id 1 network-cost 900::UNKNOWN")
...
2025-03-30 11:37:03.187  7558-7638  PeerConnection          D  onIceCandidate(candidate="1:1:candidate:1522429879 1 tcp 1518025983 ::1 52267 typ host tcptype passive generation 0 ufrag oZDr network-id 4::UNKNOWN")
2025-03-30 11:37:03.187  7558-7638  PeerConnection          D  onIceGatheringChange(COMPLETE)
2025-03-30 11:37:03.532  7558-7588  okhttp.OkHttpClient     I  <-- 201 https://api.openai.com/v1/realtime?model=gpt-4o-mini-realtime-preview (472ms)
2025-03-30 11:37:03.532  7558-7588  okhttp.OkHttpClient     I  date: Sun, 30 Mar 2025 18:37:11 GMT
2025-03-30 11:37:03.533  7558-7588  okhttp.OkHttpClient     I  content-type: text/plain; charset=utf-8
2025-03-30 11:37:03.533  7558-7588  okhttp.OkHttpClient     I  content-length: 1490
...
2025-03-30 11:37:03.535  7558-7588  okhttp.OkHttpClient     I  a=ice-pwd:nO...
2025-03-30 11:37:03.535  7558-7588  okhttp.OkHttpClient     I  <-- END HTTP (1490-byte body)
2025-03-30 11:37:03.540  7558-7638  PeerConnection          D  onSignalingChange(STABLE)
2025-03-30 11:37:03.541  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: initPlayout(sampleRate=8000, channels=1, bufferSizeFactor=1.0)
2025-03-30 11:37:03.541  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: byteBuffer.capacity: 160
2025-03-30 11:37:03.542  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: minBufferSizeInBytes: 1480
2025-03-30 11:37:03.542  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: createAudioTrackBeforeOreo
2025-03-30 11:37:03.542  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: nativeOutputSampleRate: 48000
2025-03-30 11:37:03.542  7558-7637  org.webrtc.Logging      W  WebRtcAudioTrackExternal: Unable to use fast mode since requested sample rate is not native
2025-03-30 11:37:03.549  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: AudioTrack: session ID: 289, channels: 1, sample rate: 8000, max gain: 1.0
2025-03-30 11:37:03.549  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: AudioTrack: buffer size in frames: 740
2025-03-30 11:37:03.550  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: AudioTrack: buffer capacity in frames: 740
2025-03-30 11:37:03.550  7558-7637  org.webrtc.Logging      I  VolumeLogger: start@[name=worker_thread - 7637, id=95]
2025-03-30 11:37:03.550  7558-7637  org.webrtc.Logging      I  VolumeLogger: audio mode is: MODE_NORMAL
2025-03-30 11:37:03.551  7558-7637  org.webrtc.Logging      I  WebRtcAudioTrackExternal: startPlayout
2025-03-30 11:37:03.554  7558-7638  PeerConnection          D  onIceConnectionChange(CHECKING)
2025-03-30 11:37:03.556  7558-7638  PeerConnection          D  onAddTrack(receiver={..., track=org.webrtc.AudioTrack@474a0fa, ...}, mediaStreams(1)=[...])
2025-03-30 11:37:03.556  7558-7638  PeerConnection          D  onAddStream(realtimeapi)
2025-03-30 11:37:03.557  7558-7638  PeerConnection          D  remoteDescription setSuccess()
2025-03-30 11:37:03.566  7558-7646  org.webrtc.Logging      I  WebRtcAudioTrackExternal: AudioTrackThread@[name=AudioTrackJavaThread, id=102]
2025-03-30 11:37:03.566  7558-7646  org.webrtc.Logging      I  WebRtcAudioTrackExternal: doAudioTrackStateCallback: 0
2025-03-30 11:37:06.134  7558-7647  ProfileInstaller        D  Installing profile for com.swooby.alfredai
2025-03-30 11:37:18.676  7558-7638  PeerConnection          D  onIceConnectionChange(FAILED)
2025-03-30 11:37:18.677  7558-7638  RealtimeTransportWebRTC D  +disconnect()

NOTE the ~12 seconds of nothing logged from 11:37:06.134 to 11:37:18.676.

I have reverted to month old known working code and it still does not work!

This same month old code worked less than 36 hours ago!

Notably, I also have a JavaScript implementation that I use as a test client:

It continues to work just fine using the same API key! :confused:

All I can think of is that there is something wrong w/ Android/Kotlin WebRTC client code talking to the OpenAI WebRTC server.

I have rebooted my devices.
I have uninstalled and reinstalled the apps.
I have created a new All Permissions API key.

I am nearing my wits end. :confused:

Pv

2

Looks like the Push-To-Talk feature you developed on your Android Kotlin Compose project that uses OpenAI’s Realtime API over WebRTC, has issues connecting within the past 36 hours, and also had issues in the past. The sudden disconnect, which lasts long during SDP negotiation is followed by onIceConnectionChange(FAILED) indicates that there could be something wrong with either the OpenAI Realtime API or the WebRTC client implementation. 

Reasons And Suggestions:

  1. Changes Made To Parameters Of The API:

    There have been concerns from different members of the OpenAI Developer Community regarding providing a null value to the turn_detection property during session creation. This is done with the hope of making things simple and seamless, but then the whole thing goes belly up and leads to connection failure, whereby the process of ICE remains in the state of “checking” and then ends up failing. Check your session creation parameters, especially the turn_detection which should be set to undefined or any suitable value other than null. citeturn0search1

  2. Configure the WebRTC Client:

The WebRTC client may be misconfigured leading to not connecting issues. Make sure your ICE servers are properly configured and also make sure the client is ready to deal with network change. You also need to make sure that the format and content of the SDP offer/answer are as expected.

  1. Conclusion - at least for now

Did you check for this already?
If you didn’t change any in the implementation it may definitely the problem that either a protocol, an API or API behavior or so has changed.

Maybe that helps. :slight_smile:

3

My OpenAI Realtime Kotlin WebRTC client started working again today with zero changes to my code.
I presume they fixed something on the server side.

2 Likes
4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.