I’m using Assistants and Threads and am trying to wrap my head around it.
What i’m doing is, the user makes a large request which is broken into logical segments or questions and each segment/question calls an assistant api thread to get an answer. This happens concurrently, so multiple threads are called at the same time.
I’m not sure how i’m supposed to be working with threads. What i’m currently doing is reusing an assistant for all users in all organisations (am I supposed to have one assistant for each organisation) and am then creating a new thread each time a question is asked - client.beta.threads.create. I feel like this isn’t right but am not sure what the correct approach should be. Should I continue this and then garbage collect the thread after it’s done or should I create a pool of threads and reuse them (and create a new thread and add to the pool when there isn’t enough / threads expire). What’s the right way to handle this?