Difference between streaming and not streaming when using functions

API Bugs
,
1

Hi,
I have found a fundamental difference when using tools_outputs with or without streaming.
The scenario is this:
I have found that when using streaming, sometimes I get both a message and a tools_output responses and so far I haven’t been able to reproduce it without using streaming.

Here are two screenshot images from the playground, the first - when using streaming, the second when not. All the rest is exactly the same.
Image 1:

image
image665×593 61.4 KB

Image 2:
image
image670×758 54 KB

Notice that without streaming, the assistant returned the required tool call as a text message (of code).
I thought that this may be “random” so I created a code that runs the process side by side and ran it many times. The message in the non-streaming sessions changes from run to run as expected, but the symptom remains.
Here is a colab gist with the setup instructions and the code.

The results of the run look like this:

Starting to run
Run without streaming
thread id = thread_RoNRcQglIfoRGFOvprgXKde0
message id = msg_96WY532PJK25NgyowwQ3zcPj
run id = run_Yk3Z1xH32LsBa6mEaEizu2ZG
run status= queued
run status= in_progress
run status= in_progress
run status= in_progress
run status= in_progress
run status= in_progress
run final status= requires_action
messages =  {"data":[{"id":"msg_neUyFe5WerSNqpivEnHb5deu","assistant_id":"asst_d4mvnDcYwvTN9TewV4KYegjY","attachments":[],"completed_at":null,"content":[{"text":{"annotations":[],"value":"this is a text response"},"type":"text"}],"created_at":1714650948,"incomplete_at":null,"incomplete_details":null,"metadata":{},"object":"thread.message","role":"assistant","run_id":"run_Yk3Z1xH32LsBa6mEaEizu2ZG","status":null,"thread_id":"thread_RoNRcQglIfoRGFOvprgXKde0"},{"id":"msg_96WY532PJK25NgyowwQ3zcPj","assistant_id":null,"attachments":[],"completed_at":null,"content":[{"text":{"annotations":[],"value":"Please return a text response: \"this is a text response\" and a sample call the ask_question function, ask for my age. Please return both responses to this message. Start with the text response and follow with the tool_call (ask_question) response."},"type":"text"}],"created_at":1714650947,"incomplete_at":null,"incomplete_details":null,"metadata":{},"object":"thread.message","role":"user","run_id":null,"status":null,"thread_id":"thread_RoNRcQglIfoRGFOvprgXKde0"}],"object":"list","first_id":"msg_neUyFe5WerSNqpivEnHb5deu","last_id":"msg_96WY532PJK25NgyowwQ3zcPj","has_more":false}
Run with streaming
thread id = thread_4whRrFJJKJCnnLhi72kGNx0s
message id = msg_SzAgOzm7fhbj8rRFE4ArbuKX
on_run_step_created: {"id":"step_JDotPLJlCnyFQhDT7V1xNGX4","assistant_id":"asst_d4mvnDcYwvTN9TewV4KYegjY","cancelled_at":null,"completed_at":null,"created_at":1714650958,"expired_at":null,"failed_at":null,"last_error":null,"metadata":null,"object":"thread.run.step","run_id":"run_oKZ5VGXMvGLw1PUfuzKhuRCV","status":"in_progress","step_details":{"message_creation":{"message_id":"msg_S4tycdO6pvcWjWrGR59n4u0M"},"type":"message_creation"},"thread_id":"thread_4whRrFJJKJCnnLhi72kGNx0s","type":"message_creation","usage":null,"expires_at":1714651557}
on_text_created: this
on_text_done: text = this is a text response
on_run_step_done: run step = {"id":"step_JDotPLJlCnyFQhDT7V1xNGX4","assistant_id":"asst_d4mvnDcYwvTN9TewV4KYegjY","cancelled_at":null,"completed_at":1714650960,"created_at":1714650958,"expired_at":null,"failed_at":null,"last_error":null,"metadata":null,"object":"thread.run.step","run_id":"run_oKZ5VGXMvGLw1PUfuzKhuRCV","status":"completed","step_details":{"message_creation":{"message_id":"msg_S4tycdO6pvcWjWrGR59n4u0M"},"type":"message_creation"},"thread_id":"thread_4whRrFJJKJCnnLhi72kGNx0s","type":"message_creation","usage":{"completion_tokens":7,"prompt_tokens":1771,"total_tokens":1778},"expires_at":1714651557}
on_run_step_created: {"id":"step_ZXft63Kh4kl1I82BT589wNTt","assistant_id":"asst_d4mvnDcYwvTN9TewV4KYegjY","cancelled_at":null,"completed_at":null,"created_at":1714650960,"expired_at":null,"failed_at":null,"last_error":null,"metadata":null,"object":"thread.run.step","run_id":"run_oKZ5VGXMvGLw1PUfuzKhuRCV","status":"in_progress","step_details":{"tool_calls":[],"type":"tool_calls"},"thread_id":"thread_4whRrFJJKJCnnLhi72kGNx0s","type":"tool_calls","usage":null,"expires_at":1714651557}
on_tool_call_created: tool call = {"id":"call_FPOqwmuFwyZz9QRYkdrTS8S4","function":{"arguments":"","name":"ask_question","output":null},"type":"function","index":0}
on_tool_call_done: tool call = {"id":"call_FPOqwmuFwyZz9QRYkdrTS8S4","function":{"arguments":"{\"id\":\"age\",\"text\":\"How old are you?\",\"title\":\"Your Age\",\"tooltip\":\"We are gathering this information for demographic analysis.\",\"type\":\"single_choice\",\"summary\":\"The assistant asked the user to provide their age.\",\"answers\":[{\"answer_id\":\"age_1\",\"answer_text\":\"18-24\"},{\"answer_id\":\"age_2\",\"answer_text\":\"25-34\"},{\"answer_id\":\"age_3\",\"answer_text\":\"35-44\"},{\"answer_id\":\"age_4\",\"answer_text\":\"45-54\"},{\"answer_id\":\"age_5\",\"answer_text\":\"55-64\"},{\"answer_id\":\"age_6\",\"answer_text\":\"65+\"}]}","name":"ask_question","output":null},"type":"function","index":0}
on_end

Is this behaviour a bug? Should there be a difference between streaming and not streaming?

1 Like
2

ok, I have found a problem with my implementation, the revised code is here: Google Colab
The problem is that when the run end with status =“requires_action” it has both a new message with the text and a required_action object that holds the tool cal. The problem is that this is not reflected in the playground UI.
So the api works well, it’s probably a UI issue.

1 Like