I’ve set up streaming for the Assistant API with gpt4 and it works most of the time except when the AI wants to call multiple functions. I have two event streams. The first is runs.create_and_stream
and the second is submit_tool_outputs_stream
which is nested because I assume the AI determines what functions to call in the first event stream via tools_calls
with client.beta.threads.runs.create_and_stream(
thread_id=thread_id,
assistant_id=self.assistant_id,
event_handler=EventHandler()
) as stream:
for event in stream:
# handle function calls i.e. when event == 'thread.run.requires_action' loop through tool_calls
after looping through the tool_calls and gathering the outputs. I then submit the outputs for the AI to generate a response via
with client.beta.threads.runs.submit_tool_outputs_stream(
thread_id=thread_id,
run_id=run_id,
tool_outputs=outputs,
event_handler=EventHandler()) as stream2:
for event2 in stream2:
# handle AI response
however, my second function is being called under submit_tool_outputs_stream. Is this expected? If so, then I think I would need to call submit_tool_outputs_stream recursively and risk the AI doing an infinite loop