Hi. Question about Threads. I’ve got Assistant creation up and running via the API, and I’m curious if there is any persistence in a Thread. I can create one, get and store its ID for User’s continued use at a later date.
Will that Thread maintain a list of messages exchanged between Assistant and User? If so, for how long? Indefinitely?
Or will I need to initialize it with a record of exchanges I maintain, and then continue.
Also, not clear to me. Currently, I send all prior prompts and completions with each new prompt, in order to maintain the chat context. Is this no longer required when using an Assistant via a Thread? Does the Thread keep that list for context, such that I only. need send the latest User message/prompt rather than the entire conversation to that point?
Hopefully, I’ve described the questions well enough. Thanks for any advice on this. I can experiment to determine the answers, I suppose, but perhaps someone here can answer directly. Thanks!
Yes. It’s important to store the thread ID. For whatever reason there’s no endpoint to get the list of existing threads
Yes. Not known.
For some reason only the “user” role can be manually added to messages. So you cannot modify or initialize a thread with an existing conversation. You could try and inject a summary into the system prompt. Feels hacky though.
Thanks, fellow Ronald! Some experimentation still needed, but these answers will cut me closer to the chase.
The Thread lifespan remains key (at least for my application, which requires persistence of context and exchanges over time). If the Thread maintains state then at least looks like I will not have a need to initialize with an existing conversation prior to continuing with a new message.
Not recapitulating the entire conversation with each new message should result in a solid optimization both for speed and expense.
The documentation states: " Assistants can access persistent Threads. Threads simplify AI application development by storing message history and truncating it when the conversation gets too long for the model’s context length. You create a Thread once, and simply append Messages to it as your users reply."
Just to confirm: you still need to pay for the whole conversation as if you are sending the full thing every time. The main benefit (which kind of isn’t) is that the conversation is automatically truncated for you and I guess you don’t necessarily need to maintain the full conversation.
Agreed. I’m surprised there isn’t any mentioning of thread lifespans. It could be that they are still gathering information and haven’t made a decision yet.
