# Async AssistantAPI Streaming Beta

**URL:** <https://community.openai.com/t/async-assistantapi-streaming-beta/698736>\
**Category:** API\
**Tags:** openai, beta-features, assistants-api, streaming, assistants-streaming\
**Created:** [March 27, 2024, 12:19am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736 "2024-03-27T00:19:01Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![william.zebrowski](https://avatars.discourse-cdn.com/v4/letter/w/73ab20/32.png) [@william.zebrowski](https://community.openai.com/u/william.zebrowski)\
**Post date:** [March 27, 2024, 12:19am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/1 "2024-03-27T00:19:01Z")

</div>

There are not many examples out there but curious if anyone has any luck with using the Assistants API (beta) in a async manner to push the stream to a Front End.

My applications is in python and using FastAPI as the BE Server.

Responses are taking a bit to send in full back to the user and my hope is with streaming the user will atleast start getting the response much quicker.

---

<div class="post-metadata">

**Author:** ![icdev2dev](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/icdev2dev/32/148444_2.png) [@icdev2dev](https://community.openai.com/u/icdev2dev)\
**Post date:** [March 27, 2024, 12:53am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/2 "2024-03-27T00:53:54Z")

</div>

I have NOT used async AssistantApi. Primarily because i am using Flask and while i can create a thread on which to run Aysncio, that thread does not share context with the main Flask (& too lazy to figure out how).

That said, in context of Flask, i have had to do eventlet.sleep(0) in the ’ for event in stream’…ymmv

---

<div class="post-metadata">

**Author:** ![pondin6666](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pondin6666/32/146611_2.png) [@pondin6666](https://community.openai.com/u/pondin6666)\
**Post date:** [March 27, 2024, 1:30am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/3 "2024-03-27T01:30:48Z")

</div>

For FastAPI, I ever implement streaming with chat completion successfully.

I think assistant should work as well, but one thing need to check: Do you enable gzip middleware or anything similiar?  
Just remove gzip middleware make my streaming to work. FYI.

---

<div class="post-metadata">

**Author:** ![duncansmothers](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/duncansmothers/32/243434_2.png) [@duncansmothers](https://community.openai.com/u/duncansmothers)\
**Post date:** [April 24, 2024, 5:10pm UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/4 "2024-04-24T17:10:52Z")

</div>

I’ve spent a few days working through this and haven’t fully cracked it.

There aren’t really any examples I can find so far, will come back and post if I find a good one.

---

<div class="post-metadata">

**Author:** ![brandonareid2](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brandonareid2/32/356529_2.png) [@brandonareid2](https://community.openai.com/u/brandonareid2)\
**Post date:** [May 1, 2024, 8:55am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/5 "2024-05-01T08:55:44Z")

</div>

@duncansmothers are you still looking for an example? I’ve come up with a rather basic solution using FastAPI which allows for streaming via SSE.

---

<div class="post-metadata">

**Author:** ![duncansmothers](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/duncansmothers/32/243434_2.png) [@duncansmothers](https://community.openai.com/u/duncansmothers)\
**Post date:** [May 1, 2024, 4:25pm UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/6 "2024-05-01T16:25:17Z")

</div>

Hey, yeah I’d be interested in seeing it.

I found a way to make it work via FastAPI / websockets but i’m not sure it’s the best solution.

---

<div class="post-metadata">

**Author:** ![askaraliazaruddin](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/askaraliazaruddin/32/359286_2.png) [@askaraliazaruddin](https://community.openai.com/u/askaraliazaruddin)\
**Post date:** [May 7, 2024, 8:27am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/7 "2024-05-07T08:27:59Z")

</div>

Hey @brandonareid2 , Can you share the example around streaming through SSE, would be interested to see it!

---

<div class="post-metadata">

**Author:** ![brandonareid2](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brandonareid2/32/356529_2.png) [@brandonareid2](https://community.openai.com/u/brandonareid2)\
**Post date:** [May 9, 2024, 12:58pm UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/8 "2024-05-09T12:58:22Z")

</div>

I created a medium article walking through the implementation, but for some reason I’m not allowed to post links here.

You can find in my profile a link to my medium. At the bottom of that article is a link the the example repository.

---

<div class="post-metadata">

**Author:** ![limhss](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/limhss/32/378207_2.png) [@limhss](https://community.openai.com/u/limhss)\
**Post date:** [May 23, 2024, 1:19am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/9 "2024-05-23T01:19:39Z")

</div>

@brandonareid2  
Got it to work with my Telegram messagehandler. Telegram user is able to get questions asked about the files attached to the assistant. I tried to incorporate functions. However I could not figure out how to get it to work. I got as far as to add on\_event. While I used api to send the output via assistant api and examine the stream response, I could see the message created by ChatGPT but not stream it out to Telegram. Will you be working to enhance your solution support functions?

Currently, I am calling the submit\_tool\_outputs API with stream=True via a requests.post and parse the string to find “thread.message.completed” for the associated data and issuing self.queue.put\_nowait(data[‘content’][0][‘text’][‘value’])

---

<div class="post-metadata">

**Author:** ![jhakulin](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jhakulin/32/449110_2.png) [@jhakulin](https://community.openai.com/u/jhakulin)\
**Post date:** [May 24, 2024, 4:10am UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/10 "2024-05-24T04:10:59Z")

</div>

Here is one basic example for streaming with OpenAI assistant using SSEs and Quart application as backed and JS frontend  
[azureai-assistant-tool/samples/FileSearch at main · Azure-Samples/azureai-assistant-tool (github.com)](https://github.com/Azure-Samples/azureai-assistant-tool/tree/main/samples/FileSearch)

---

<div class="post-metadata">

**Author:** ![kinana](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kinana/32/146267_2.png) [@kinana](https://community.openai.com/u/kinana)\
**Post date:** [July 16, 2024, 11:48pm UTC](https://community.openai.com/t/async-assistantapi-streaming-beta/698736/11 "2024-07-16T23:48:19Z")

</div>

Some medium articles cover the topic but i found them to be too complex for what i need, you can search Meduim and take a look to check if the solutions there better fit your use case.  
After looking around a bit though, i found a simpler way to achieve streaming using SSE with FastAPI with the following example snippet:

```auto
async def stream_assistant_response(thread_id, assistant_id):
  
    client = AsyncOpenAI()

    stream = client.beta.threads.runs.stream(
        thread_id=thread_id,
        assistant_id=assistant_id,
    )
    async with stream as stream:
        async for text in stream.text_deltas:
            yield f"data: {text}\n\n"

```

Make sure you also set the media\_type correctly in the API response logic:

```auto
return StreamingResponse(
        stream_assistant_response(thread_id, assistant_id), media_type="text/event-stream"
    )

```

this code works fine for me when i test the endpoint on postman, i am able to see every event with it’s corresponding text\_delta.  
not sure if there are underlying issues with the logic that i can’t catch atm.

you can find the references that were helpful for me in the helpers.md file in OpenAI’s python API library repo, and in the issue #1473 on the repo as well.  
Hope that helps!
