Realtime API using SIP - Call dropping after 1 second - BYE from OpenAI

API
1

Hi,

I’m building a PoC using Twilio trunks but experiencing calls dropping after 1 second. I’ve tried changing my /answer payload to be as simple as possible incase there was an issue with this:

{
“type”: “realtime”,
“model”: “gpt-realtime”,
“instructions”: “You are Fred, a friendly concierge for the day.”
}

Checks I’ve carried out:

  • Twilio trunks are configured to use TLS & SRTP

  • BYE from OpenAI has a cause code of 16 (not helpful)

  • Alaw is offered by the codec

  • Twilio’s portal shows the call with no codec

    image
    image2290×1046 105 KB

From my understanding (from reviewing the community and api docs) the websocket connection is optional and not required for the call to stay connected.

Can anyone shed any light on the issue that I’m experiencing? I’ve seen other similar issues posted with asterisk PBXs but I’m using Twilio as shown in the examples.

Thanks

2

I added the websocket just in case it was required, but I get a 404 when trying to connect.

=== Incoming webhook ===
{
“id”: “evt_68efc661bcdc819080a4ddf6c0275cf3”,
“object”: “event”,
“created_at”: 1760544353,
“type”: “realtime.call.incoming”,
“data”: {
“call_id”: “rtc_596c8809059942fb9f701f6a96d3f930”,
“sip_headers”: [
{
“name”: “Record-Route”,
“value”: “sip:54.171.127.192:5061;r2=on;lr;transport=tls”
},
{
“name”: “Record-Route”,
“value”: “sip:54.171.127.192;r2=on;lr”
},
{
“name”: “From”,
“value”: “sip:+4433xxxxxxxxxxxx@pstn.twilio.com;tag=63636200_c3356d0b_e740ba0b-7cc6-484a-ad41-2fa6fd85e467”
},
{
“name”: “To”,
“value”: ““sipout” sip:proj_7SIFxxxxxxxxxxxxxxxxxxxxxxxxxx@sip.api.openai.com;transport=tls;tag=33bfa43c-f821-4cbc-8b6b-e975b770adfb”
},
{
“name”: “CSeq”,
“value”: “592657 INVITE”
},
{
“name”: “Max-Forwards”,
“value”: “63”
},
{
“name”: “P-Asserted-Identity”,
“value”: “sip:+4433xxxxxxxxxx@151.2.143.17:5060;user=phone”
},
{
“name”: “Diversion”,
“value”: “sip:+441615xxxxxxx@twilio.com;reason=unconditional”
},
{
“name”: “Call-ID”,
“value”: “5b70c5ad757b37ce47f323895761b548@0.0.0.0”
},
{
“name”: “Via”,
“value”: “SIP/2.0/TLS 54.171.127.192:5061;branch=z9hG4bKcb09.e5a7fdf55f349af6f98baed66534b1b0.0”
},
{
“name”: “Via”,
“value”: “SIP/2.0/UDP 172.18.197.19:5060;rport=5060;branch=z9hG4bKe740ba0b-7cc6-484a-ad41-2fa6fd85e467_c3356d0b_584-15419146977853018054”
},
{
“name”: “Contact”,
“value”: “sip:+44330xxxxxxxxxxxx@172.18.197.19:5060;transport=udp”
},
{
“name”: “Allow”,
“value”: “INVITE,ACK,CANCEL,BYE,NOTIFY,OPTIONS,REFER”
},
{
“name”: “X-Twilio-AccountSid”,
“value”: “AC9ea30bf26c51897xxxxxxxxxxxxxxxx”
},
{
“name”: “User-Agent”,
“value”: “Twilio Gateway”
},
{
“name”: “Content-Type”,
“value”: “application/sdp”
},
{
“name”: “X-Twilio-CallSid”,
“value”: “CAfc00ecd22dcb39db92484846f91267f4”
},
{
“name”: “Content-Length”,
“value”: “362”
}
]
}
}
Sending /accept with payload: {
type: ‘realtime’,
model: ‘gpt-4o-realtime-preview’,
instructions: ‘You are a friendly phone receptionist that answers incoming calls.’,
modalities: [ ‘audio’, ‘text’ ],
input_audio_format: ‘wav’,
output_audio_format: ‘wav’,
voice: ‘verse’,
input_audio_transcription: { model: ‘whisper-1’ }
}
=== /accept response === 200
:link: Attempting WebSocket connection to: wss://api.openai.com/v1/realtime?call_id=rtc_596c8809059942fb9f701f6a96d3f930
:stopwatch: WebSocket still not open after 2s — likely handshake issue
:warning: WebSocket error: Error: Unexpected server response: 404
at ClientRequest
:cross_mark: WebSocket closed — code: 1006, reason:

Feels like an issue with RTP between Twilio and OpenAI, but due to the use of SRTP Twilio can’t provide signalling.

Feels like some CDR type events in OpenAI are really required to build a production service on this endpoint.

Can anybody from OpenAI check out what they see at their end?

3

I realised I was using the webhook secret rather than than a API key associated with the project.

I got a 200 even thought the authentication was failing.

1 Like