Hi everyone,
I’m new to the assistants-API world and want to ensure I understand how Threads work because it has become confusing for me.
My assistant_v1 has a prompt X.
If I create a new thread and run it with assistant_v1, the prompt for this thread will be prompt X.
I updated my assistant_v1 prompt to prompt Y.
However, when I create a run in my old thread, it generates a message as if prompt X is still the current one. On the other hand, new threads generate responses based on prompt Y.
Is this understanding correct? Do I need to manually update something in my runs/threads every time I execute them?
This issue is bothering me because I’ve been using a 1 Thread per User architecture to maintain a full conversation history. But now, with this behavior, I feel forced to create a new thread every X period of time.
Can anyone help me with this? Or at least explain how it works properly?
A thread represents a conversation session between user and assistant.
An assistant ID has “instructions”, which is the first privileged instruction inserted before this chat replay sourced from threads. It is placed as a model’s “system” role.
A thread with a conversation will have an element of self-training to it. An assistant role that has been answering as a children’s monkey character will not pivot 100% to an industrial engineering expert by changing the Assistant ID. You might use a generic instructions, and the AI still maintains monkey chat.
Therefore, you may want to force a new conversation if the instructions or tools need a significant change. Disallow mid-chat personality switches by new Assistant ID.
(You should not be running a thread again without adding another user message for the AI to answer.)
I only run the threads again when a user send a new message. This trigger my thread and run it with the new message inserted.
Ur saying the thread messages gonna track the same pattern as they were being sent before? Even the new prompt giving more info or new rules abt how the new messages need to be formatted?
Yes, AI models are pattern-followers. The quality of answering based on a system message and a latest user input, and demoting attention to contradictory assistant messages or even assistant tool use will depend on the AI model, its attention weighting, and how much “chat” training it has to follow or even disregard system or other messages (over assuming a typical “I’m ChatGPT” chat entity).
“Chat history” can also be “multi-shot examples”. A task can be completely trained by examples instead of instructions, especially on completion AI (the playground for which has been rudely removed).
Pictured below is GPT-4o, with no system message but a monkey chat before the latest input. The latest question is not answered as it would be with no conversation history…
Got it, but let’s say this:
My assistant had in prompt X: “Ask for the date in this format dd/MM.”
Now my assistant has in prompt Y: “Ask for the date in this format dd/MM/yyyy.”
It should use the prompt as the main rule, even if the chat history creates a base context that is appended with the prompt. If I want to control the assistant’s behavior based on the prompt, I’ll have to create a new thread every time my assistant is updated.
For clarity, you need to never use the word “prompt” again! Prompt can be the entire API input placed into context window that inspires the AI model to generate, or mean many other things to different people.
It is a term that comes from completion AI, where you would fabricate a “writing prompt” like “America’s favorite fruit is” to have the production of the desired next tokens, “bananas. Over 100 million…”
Assistants takes an instruction field. This is placed as the first system role message in a chat-completions style list of messages.
Threads does not contain the instruction. It only contains user messages and assistant responses (or sometimes between those, assistant tool calls and tool responses).
So you can freely change the instruction, or append to it. Or make it stronger, and see how using a forceful instruction disrupts any possible pattern learning from prior responses:
“This mandatory and permanent date format for writing date requests to the user overrides *any* previous usage seen before the latest user input: `dd/MM/yyyy`”
(the AI doesn’t have to know the previous instruction was also called “permanent”…)