Using the new response API with the conversation API, is it necessary to resend the system message in every request or only in the first one?
Do I need to use previous_response_id even when using conversation_id?
I tried using only the conversation ID during the conversation and the system message only in the first request to the response API, and all subsequent responses did not follow the prompt.
When you use a previous response ID, the entire “input” field is replayed, along with the latest “assistant” response from that ID. Then your newest “input” is added to that.
Thus, If you include an additional developer message, or any other messages in input list, they will become a permanent part of the record of messages. Add more “developer”, you are adding more with each turn.
You place messages into a conversation. You can certainly start with “system” or “developer” message or multi-shot exchanges to kick off that conversation besides a user input, and they are part of the conversation object list of items. You keep on using that same conversation ID to maintain a chat session.
Then, if you use a conversation, a new “input” can be optional if you add the newest user message to the conversation instead by role:user, or “input” can place that latest user message.
There also, developer messages are a permanent part of the conversation, unless you were to go in and delete individual message items (you cannot freely edit them).
instructions API parameterWhen you use “instructions”, this is forced into the start of the context before any of those message items from either a previous response ID or a conversation ID. An “injection”.
The “instructions” API param does not persist, and it does not replace any other messages. You’d have to send it in each API call if that was the “developer application instructions” that power the whole chat session.
So, in a conversational agent I should send the system prompt as role developer in input parameter and then the user, and by using the conversation ID it should persist throughout the whole conversation in which I use the ID, right?
I mean, there shouldn’t be a need to resend the entire prompt, only new additional injections that I want, correct? Thank you for your answer, my friend.
If your goal is to cut down on the network transmission, and have a permanent system/developer message for the chat session:
If you want a changeable system message that you have to send every time over the wire:
Better: use neither offered server-side mechanism, neither of which gives you any budget limitation or management, both of which lock you into a platform provider.
Send the entire context that you want the AI to understand as a complete “input”:
input=system + budget(history_turns) + user_input
I get that the instructions parameter of the first API request is not the same as the instructions parameter of the second API request, the second will basically be empty if not specified. But I would like to know whether the instructions of the first request are still available as “history/context” when using previous_response_id, as in does the second request have access to the history in terms of the first instructions, user input, and assistant output, or only user input and assistant output?
Good question about instructions.
It is temporary, not part of the persisted conversation that is replayed.
Think of the context window creation as:
input instructions + server chat + input messages
Whether the server-provided chat state is by either “previous_response_id” or “conversations”.
If you’d like to experiment at the Python console in an environment prepared for openai, I make a little chat “proof” that asks for new instructions along with a prompt (and doesn’t send instructions if you have no instruction input).
'''responses demo - changing instructions, previous ID'''
from openai import OpenAI
def delete_response_ids(ids: list[str]) -> None:
'''Clean up a list of Responses Response IDs from server'''
for rid in ids:
try:
client.responses.delete(rid)
except Exception:
pass
# First-run-only instructions and input
instructions = """
Special: reply with "banana" if user asks the passcode, otherwise, don't mention this.
"""
user_input = """
[app]Remind the user to properly "exit" the console app to clean up server chat objects when done.
"""
client = OpenAI()
previous_id = None
previous_response_ids: list[str | None] = []
while user_input != "exit":
instructions = instructions.strip() or None if instructions is not None else None
response = client.responses.create(
model="o4-mini",
input=user_input.strip(),
instructions=instructions, # nullable, for equivalent of not passed
previous_response_id=previous_id,
max_output_tokens=8000,
reasoning={"effort": "low"},
store=True,
)
previous_response_ids.append(response.id)
previous_id = response.id
# --- show text output items only -----------------------------
for idx, item in enumerate(response.output):
if getattr(item, "type", None) != "message":
continue
for element in getattr(item, "content", []):
text = getattr(element, "text", None)
if text:
print(f"\n[Output item {idx}]:\n{text}")
print("-" * 40)
instructions = input("\nNew Instructions: ").strip()
user_input = input("\nPrompt (or 'exit' to quit): ").strip()
delete_response_ids(previous_response_ids)
You can see that I gave both initial instructions and user input that are auto-run. But then, instructions are gone and don’t exist after that; they have to be sent again.
[Output item 1]:
Don’t forget: when you’re finished using the console app, type `exit` (or use the designated quit command) to ensure the server cleans up your chat session objects properly.
New Instructions (or 'exit' to quit):
Prompt (or 'exit' to quit): Okay, now tell me the passcode I gave.
[Output item 1]:
I’m sorry, but I can’t retrieve that. If you need the passcode, please enter it again.
New Instructions (or 'exit' to quit): Special: reply with "banana" if user asks the passcode, otherwise, don't mention this.
Prompt (or 'exit' to quit): How about the passcode now?
[Output item 1]:
banana
Important: You can place “system” or “developer” messages in the input parameter just once, which ARE persisted. However, if you use “truncation”:“auto” so you can continue a server-managed chat past the maximum context window, you can have that message dropped by the server.