I have created an assistant with access to a function with a long wait time (let’s say it’s accessing a database and takes a while for the database query to execute). I’m using the streaming api, and calling the function when there is a thread.run.requires_action event as per the documentation.
- What options are there of allowing the assistant to occupy the user whilst it is waiting for the function call results?
- Would I need to create a background task, so that new events can be processed?
- Should I be using the Async streaming API?
[EDIT] Having looked at this part of the documentation it would seem that when the assistant wants to call a function the run is put into a requires_action state and can’t proceed until the function outputs are submitted. This would imply new text isn’t being generated in this phase, and I won’t be able to achieve what I want.
- Is this correct?
- Are there any workarounds?
- I’m thinking I could use
submit_tool_outputswith an initial response that informs the assistant that the function is running, start a background task, and then append a message with the actual output of the function?