Benchmarking Tools Again - SO SLOW

[ollie2]

My testing, which I’ve posted extensive results in the below thread, also gave me the impression assistants isn’t suitable for production use. I can’t have my users waiting 10-35 seconds for a response.

Assistant/Thread Model Stress Test: Concerning Results [See inside]

[Results graphs at bottom, see below] I’m designing an app where I’m thinking of having one assistant and one thread per user. The assistant represents a chat bot that users of the app chat to. Each user has their own thread. One thing that’s important to me is low latency of the chat bot response, so I ran a test measuring the response time of the bot (the time taken of the run; the invocation of the assistant on a user’s thread that returns a text response) over time as the thread increases i…