Streaming with Assistants API

API
,
1

I’m building an assistant and currently using the following code:

client.beta.threads.runs.create(
    thread_id=thread_id,
    assistant_id=assistant_id,
    model=model,
    temperature=temperature,
    stream=True
)

However, after reading the OpenAI documentation, specifically here: https://platform.openai.com/docs/assistants/tools/function-calling, I noticed they use:

client.beta.threads.runs.stream(
    thread_id=thread.id,
    assistant_id=assistant.id,
    event_handler=EventHandler()
)

The first thing I observed is that with the second method, I have to use a with statement on the returned object, whereas in the first method, I can simply iterate over it. Also, with the second method, I can pass a handler.

My questions are:

  • What are the real differences between the two approaches?
  • Which one is better?
  • Is one more suitable for production software?
  • Is it best to use the handler as in their example? On the handler they use the on_event() method, but it’s tricky to pass parameters to this method since it’s automatically called when a new event arrives.
4 Likes