Having trouble with Assistant API function calling streaming

API
1

After using the provided code int the documentation for the Assistants API streaming function calling, If the model does not choose to call the function it does not print.

The reason is here

def on_event(self, event): # Retrieve events that are denoted with 'requires_action' # since these will have our tool_calls if event.event == 'thread.run.requires_action': run_id = event.data.id # Retrieve the run ID from the event data self.handle_requires_action(event.data, run_id)
However I don’t know how to make it answer in streaming without the function call.

2

extend your on_event function with other events related to text response like “thread.message.created”, “thread.message.in_progress”, “thread.message.delta”, “thread.message.completed”.

thread.message.delta will contain the text chunks as it is streamed, you want this.

thread.message.completed will contain the complete text response.

4

Thank you! Would you give me that chunk of code please? When I try to this it prints a duplicate.

InIn San San Francisco Francisco today today, the the current current temperature temperature is is 5757°F°F… The The probability probability of of rain rain is is 66%.%.

5

There’s a pretty thorough example here: https://platform.openai.com/docs/assistants/overview

6

Still cant figure it out can you guys please give me the code?

7

if you extend your original code:

def on_event(self, event):
    if event.event == 'thread.run.requires_action':
        run_id = event.data.id
        self.handle_requires_action(event.data, run_id)
    elif event.event == 'thread.message.delta':
        // event.data.delta.content[0].text.value
    elif event.event == 'thread.run.completed':
        // text response complete

you probably have the same code already. as for the repeating texts, it is probably something with your streaming handler unrelated to the API. try to print to console the output when you receive the delta and see if it is repeating (most probably not).

8

Follow this source code to do it?

9

I do have the same code, could you give me yours that is working?

10

That is in js and I need python.

11 
class EventHandler(AssistantEventHandler):
    @override
    def on_event(self, event):
        if event.event == 'thread.run.requires_action':
            run_id = event.data.id
            self.handle_requires_action(event.data, run_id)
        elif event.event == 'thread.message.delta':
            sys.stdout.write(event.data.delta.content[0].text.value)
            sys.stdout.flush()
        elif event.event == 'thread.message.completed':
            sys.stdout.write('\rAssistant: ' + event.data.content[0].text.value)
            sys.stdout.flush()
        elif event.event == 'thread.run.completed':
            print("\nRun completed")
            
    def handle_requires_action(self, data, run_id):
        tool_outputs = []
        print("Handling required action")
        for tool in data.required_action.submit_tool_outputs.tool_calls:
            if tool.function.name == "get_current_temperature":
                tool_outputs.append({"tool_call_id": tool.id, "output": "57"})
            elif tool.function.name == "get_rain_probability":
                tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})
        
        # Submit all tool_outputs at the same time
        self.submit_tool_outputs(tool_outputs, run_id)
 
    def submit_tool_outputs(self, tool_outputs, run_id):
        # Use the submit_tool_outputs_stream helper
        print("Submitting tool outputs")
        with client.beta.threads.runs.submit_tool_outputs_stream(
            thread_id=self.current_run.thread_id,
            run_id=self.current_run.id,
            tool_outputs=tool_outputs,
            event_handler=EventHandler(),
        ) as stream:
            for text in stream.text_deltas:
                print(text, end="", flush=True)

def message_assistant(thread_id, message):
    client.beta.threads.messages.create(
        thread_id=thread_id,
        role="user",
        content=message,
    )
    with client.beta.threads.runs.stream(
        thread_id=thread_id,
        assistant_id=assistant.id,
        event_handler=EventHandler()
    ) as stream:
        stream.until_done()

thread = client.beta.threads.create()
print("Thread created for session")
message = "What is the current temperature in San Diego, CA?"
message_assistant(thread.id, message)

Here is my code, but I get this weird output.
The current temperature in San Diego, CA is 57°F. Diego Diego, CA CA is is 5757°F°F…

1 Like
12

can anyone please help me?

13

Why don’t you ask chatgpt to tranlsate my javascript code to python with the same logic ? The code I pasted is guaranteed to be working