How to Resume Streaming in Python After Submitting Function Call Outputs in OpenAI Assistants API?

API
1

Subject: Streaming Stops After Function Call Response Submission

Hi,

I am trying to use function calls with streaming. I can successfully stream responses up until the requires_action event, after which I submit the function tool call response.

This submission happens successfully, and if I print all messages, I can see that the assistant does reply after the function call result is submitted.

However, the stream does not continue to output the GPT response, and it stops after submitting the function call result.

Does anyone know how to make the stream continue and capture the assistant’s full response after the function call execution?

Steps to Reproduce

Below is my implementation:

from openai import OpenAI
import os
from dotenv import load_dotenv
import requests
import json  # Added for handling JSON

load_dotenv()
client = OpenAI()

assistant = client.beta.assistants.retrieve("asst_oa8dz4R0M9XBWdOqmxq22Lnp")
thread = "thread_bG77CUSW5xqWE3PfHXcJIZW0"

message = client.beta.threads.messages.create(
    thread_id=thread,
    role="user",
    content="Can you please use both Code Interpreter and the multiplication tool (not the multiplication tool twice, one of the tries has to be with Code Interpreter) to find 1111 times 1111?"
)

print("Starting run stream...")

stream = client.beta.threads.runs.create(
    thread_id=thread,
    assistant_id=assistant.id,
    instructions="Please address the user as Jane Doe. The user has a premium account.",
    stream=True,
)

def multiply_two_numbers(a, b):
    return a * b

stream_content = [] 
for event in stream:
    stream_content.append(event)

    if event.event == "thread.message.created":
        print("\nAssistant:")
    elif event.event == "thread.message.delta":
        print(event.data.delta.content[0].text.value, end='')
    elif event.event == "thread.run.step.created":
        if event.data.step_details.type == "tool_calls":
            print("\nTool Call:")
    elif event.event == "thread.run.step.delta":
        if hasattr(event.data.delta.step_details.tool_calls[0], "code_interpreter"):
            print(event.data.delta.step_details.tool_calls[0].code_interpreter.input, end='')
        else:
            if event.data.delta.step_details.tool_calls[0].function.name:
                print(f"Calling function {event.data.delta.step_details.tool_calls[0].function.name}")
            print(event.data.delta.step_details.tool_calls[0].function.arguments, end='')

    elif event.event == "thread.run.requires_action":
        tool_outputs = []
        current_run = client.beta.threads.runs.list(thread_id=thread).data[0]
        
        for tool in event.data.required_action.submit_tool_outputs.tool_calls:
            function_name = tool.function.name
            arguments = json.loads(tool.function.arguments)
            
            if function_name == "multiply_two_numbers":
                multiply_result = multiply_two_numbers(arguments.get("a"), arguments.get("b"))
                tool_outputs.append({
                    "tool_call_id": tool.id,
                    "output": str(multiply_result)
                })

        # Submit all tool outputs at once after collecting them in a list
        if tool_outputs:
            print("\nThe tool output is:")
            print(tool_outputs)
            run = client.beta.threads.runs.submit_tool_outputs(
                thread_id=thread,
                run_id=current_run.id,
                tool_outputs=tool_outputs
            )
            print("Tool outputs submitted successfully.")

# Function concludes with 'thread.run.step.completed'

Observed Behavior

  • Before the function call (requires_action event):
    • Streaming works as expected.
  • After submitting the function call response:
    • The assistant correctly processes the function output and replies.
    • However, the stream stops, and I do not see the assistant’s response in real-time.
  • If I fetch all messages manually, I can see the assistant’s reply.

Expected Behavior

  • The stream should continue past the function call and display the assistant’s response after processing the function result.
  • ChatGPT’s UI does not stop streaming after function calls, so the API should also support this.

Question

How can I ensure that the stream continues after submitting the function call results?

Any help would be much appreciated!

Thanks!

2

Creating Streams

There are three helper methods for creating streams, a new paradigm from what you demonstrate in your code:

client.beta.threads.runs.stream()

This method can be used to start and stream the response to an existing run with an associated thread that is already populated with messages.

client.beta.threads.create_and_run_stream()

This method can be used to add a message to a thread, start a run and then stream the response.

client.beta.threads.runs.submit_tool_outputs_stream()

This method can be used to submit a tool output to a run waiting on the output and start a stream.

This also takes an understanding of the events needing to be collected or parsed from a stream.

1 Like