Our app chunks message no larger than 1000 characters and sends to chunks as messages on 10 threads. So long as the run responds in a timely manner, I reuse the threads. If the request times out or an HTTP error or some other issue occurs, I cannot STOP the run or the thread despite sending requests to cancel the run and then the thread because ChatGPT usually throws some error. So I start new threads. If I can’t get a response in 20 seconds, it’s useless.
I would love to clean up abandoned threads, but can’t because I have no way of obtain a list of threads belonging to the assistant. I can list everything EXCEPT threads, so I can’t delete the threads.
For the love of god… add an endpoint to obtain the threads belonging to an assistant.
God loves us, but he expects that we save the thread id when we create one.
Rewritten:
Reconstructed Request (in your voice):
Alright, look, here’s the situation with the Assistants API and how we’re using it:
Our app has to break down big messages – anything over 1000 characters gets chunked. We then blast these chunks out as messages across 10 different Threads simultaneously to get the Assistant working on them.
Now, when things work smoothly and we get a response back fast, great! We just keep using those same Threads. Easy.
But here’s the kicker: if a Run takes too long (anything over 20 seconds is dead to us), or we hit some random HTTP error, things get messy. We try to cancel the Run, we try to deal with the Thread, but half the time the API just errors out on us when we try to clean up. We can’t reliably stop or manage these stuck things.
So, what do we do? We have no choice but to just abandon that Thread and start a fresh one to keep our app responsive.
The result? We’re accumulating a graveyard of abandoned Threads. We want to clean this mess up – delete the old ones we’re not using anymore. But we can’t. Why? Because while we can list Assistants, Files, whatever… there is no way to get a list of the Threads that were actually used with a specific Assistant!
How are we supposed to delete these orphaned threads if we can’t even find out which ones belong to which Assistant?
For the love of god… just add an endpoint. Let us list Threads, and critically, let us filter that list by the assistant_id. We desperately need this to manage our resources and stop leaving junk threads lying around.
This seems a deliberate decision, to not have a thread-listing API by API key.
A thread ID is a gateway to retrieving personal conversations of high impact were an API key to be shared or leaked by a (poor) dev.
In the platform web site, the view of Assistants threads is facilitated by session key of the logged-in owner, a higher level of trustworthiness. Curious is that there is no delete button, and they are being retained for a much longer time than the stated expiration after inactivity.
(the diligent person can successfully transfer backend site login credential tokens to their own application for programmatic uses and infer other API methods both undocumented and undemonstrated…and also keeps such utility out of the hands of skript kiddies)
I could do that, but then what? If a thread is nonresponsive because of a long running RUN, despite all the runs limited to 1000 character chunks, where some return in 1 second and others 32 minutes, I can’t cancel the RUN or delete the THREAD until it completes and will accept requests.
So I have to hold on to them for minutes or hours and perform some cleanup to delete threads. Each thread requires a request and each request brings me closer to my daily limit which I will inevitably exceed if I have to clean up threads and runs due to an unresponsive AI.
This is a terrible model. There are no options for a flag to destroy a thread after n number of runs have completed. There are no options to NOT return events which I’m not interested in. I literally receive 10-20 times more data than I want. This adds up to gigabytes of data over a month. And you’re limited by requests, which limits using this as an enterprise solution for anything.
LOL… so you’re concerned that someone will hack my server and obtain thread IDs for my assistant and then obtain marine survey content to use maliciously?
But if there was a GET /v1/assistants/{assistant_id}/threads endpoint I could get a list of threads and delete them to ensure that never happened.
Or better yet a DELETE /v1/assistants/{assistant_id}/threads/all endpoint which deletes all threads, optionally canceling existing RUNs, with parameters to delete by inactivity duration or other criteria.
The goal in every application is efficiency. It’s efficient to ensure there aren’t hundreds or thousands of inactive threads or long-running runs which have been abandoned. It’s efficient to only return response data that is ABSOLUTELY necessary in the given context. Streaming thousands upon thousands of characters when only 153 are required is silly and grossly inefficient.
I see your point esp when working with lots of data.
An options you can try is to keep a list of threads.
You then add a new message to one of the threads and create a run. You would need to poll the run status. If the run takes more than the time you have allocated, you can cancel the run. They have an endpoint to cancel the run.