Realtime Websocket Confusion

API
1

Hey everyone! I’m working on adding a real-time ChatGPT integration for my project Homeway. But I’m hitting some odd issues I don’t think the docs explain very well.

For context, I’m using the official c# SDK and have followed the real-time example in the repo. I’m using 4o-mini and text only.

Occasionally, the real-time response will have no items created in the response.

I logged every event sent back and saw ResponseStarted and then ResponseFinished with no other events. In the working scenario, I see the item-created events, delta streaming, etc. I also checked the ResponseFinished object; its created items array is empty. I’m sure I added a user message and then sent the start reply message, so I have no idea why it’s responding with nothing.

Proper Use Of The Websocket API

I assumed since I created a session and the WebSocket is active, I wouldn’t have to resend all of the past messages as I do with the chat completion REST API. But it seems that’s not the case. It looks like the instructions are applied to every message, but it seems unless I write each user and assistant message every time, the response has no memory of the past conversation.

Is that expected? I can’t find anything about the expected behavior on the website API docs or in the SDK.

If anyone has any insight, I would love to hear it!

2

Hi and welcome!

There is a github repo here that has a working example in code with WebRTC

Also one in python and websockerts here

3

Thanks for the reply! I looked at those two repo, and they are helpful, but they aren’t quite the same thing. I have found a few voice demos, but none just text. What I’m really looking for is what the expected event flow is for a text-only conversation over the real-time web socket.

In the open AI docs they document the web socket endpoint and the events, but there’s nothing I can find on the expected even flow etc.

4

Thanks for sharing this problem, i am also facing this same issue.

6

I’m also seeing random error messages returning with no context beyond “Kind”:28. I looked at the docs but don’t see error definitions.

I know it’s just a beta API, and it’s nice for what it is. I hope this feedback helps build the docs to make them more useful.

7

I put this in another thread but I share your frustration:

I’m using the realtime javascript sdk per your examples.

import {RealtimeAgent, RealtimeSession} from '@openai/agents-realtime';

  // ... stuff not worth pasting here ....

  // relevant parts....
  const agent = new RealtimeAgent({
    name: 'realtime',
    instructions: instructions.value,
    prompt: {
      promptId: 'pmpt_68bf460f699c8195a76e1a48a01473620fce5dc25a0963aa',
      version: '10',
      variables: {
        patientname: 'John Smithj'
      }
    } as any
  })

  session = new RealtimeSession(agent, {
    model: 'gpt-realtime',
  })
  await session.connect({apiKey: apiKey.value})
  1. The promptId is ignored. It wasn’t until I turned on error logging that I found what I think is the problem. I’m using your online prompt editor and gpt-realtime is not an option so this call silently fails because gpt-5 does not match gpt-realtime
  2. I get an error that patientname is not an object
  3. It is crazy hard to get to the data channel. I want to send events and it’s not clear at ALL how to do that. Looking over your source the data chanell I would use is not exposed.

Basically it seems impossible to use the existing higher level realtime API for anything real.

I should add that this helped me alot

  session.on('transport_event', (event) => {
    console.log('transport event', event);
  })