I am looping over a bunch of prompts, however, I only need the conversation history of the recent 3 prompts not the whole conversation with the thread. If I prompt over 100 prompts assistant will append the whole conversation which inflates my cost. I was wondering if I could create a new thread after the third prompt and avoid appending the whole conversation history that way.
Does creating threads come at a cost?
If I create numerous threads let’s say a 1000 will this add to the cost of the assistant?
I am trying to avoid appending the whole conversation history, i just need the thread to know about the three recent prompts only.
You have a logical way to defeat a lack of logic. However, it is almost as if OpenAI anticipated that what you describe could be useful for shortening a chat session – and then blocked adding assistant role messages to a thread, so you cannot recreate a conversation at shorter length.
You can have limited success with a user message including “here’s what we’ve been talking about…” or “here’s my conversation with another AI”.
Threads don’t cost if idle. Only in the odd case of a thread being left unfinished, such as waiting for you to return a function, might you then be going into undocumented territory - a message file still “attached” to an assistant (or a way to not pay for attaching).
1 Like
Well… I guess only those using the Community-maintained PHP lib are having a good time deleting messages from the thread. I’ve been doing this when I still had the energy to endure Assistants API. It finally succeeded in sending me back to Chat Completion. Good luck everyone with being charged for this BETA stuff. I see they started charging code_interpreter usage on the 13th… and the AI will simply impress itself in utilizing that the moment even if it needs to solve 1+1. The only drawback to the message delete is that it for some reason doesn’t delete the function tool call & responses
1 Like
Yes, deletion is another technique that I overlooked, a previously undocumented feature that you can’t access with a library (now blocking by pre-validation).
When you are storing the conversations on your side to track when they should be deleted (and still present the user with their chat), it makes one more premise for use of an assistant redundant.
1 Like
Thank you for the response. Also, do I need to delete the Assistants I created? Is there a daily charge for keeping the Assistants?
“Assistants” (you know, GPTs have a three letter abbreviation - so should these…) will incur daily charges if they have files attached. $0.20 per day after the free period (which has been extended) from memory. So check the guide for them to find the latest pricing for both that and for code interpreter instances per thread.