Integrate Assistant API Streaming through websocket

API
, ,
1

I understand AssistantsAPI Streaming is relatively new and as I’ve been digging, there are not many examples out there incorporating this feature with a front end chat widget but curious if anyone has had any luck with this?

For this streaming to occur real-time, I’m assuming a web socket has to be in place?

If any examples are out there for python that would be great - if you have another language, I would be happy to take it and convert it.

Thanks!

1 Like
2

You can stream the response of the create run API from your server and read the streamed data using the fetch API on the client side. No need of websockets.

1 Like
3

Thanks @cloudUser98. Does this option assume that we get the response in full first and then iterate through the characters in the response to output to the user?

Or is your option streaming the response in real time?

Thanks in advance for the clarification!

1 Like
4

My answer was about streaming the API response in real time.

Let me show you an example for my approach using python, the openai python SDK and the Django framework.

    from openai import OpenAI
    client = OpenAI()

    stream = client.beta.threads.runs.create(
        thread_id="thread_123",
        assistant_id="asst_123",
        stream=True
    )

    def my_iterator():
        for event in stream:
            yield event

    return StreamingHttpResponse(my_iterator())

this is only an example as you would need to add more code so you can extract the text value from each stream event.

Then you can read the stream of data using the javascript Streams API on the client side.

1 Like
5

See my topic here…One Shot Streaming Using Svelte with Bi Directional Web Sockets

1 Like