Multiple system messages?

API
1

The Chat Completion docs say:

Typically, a conversation is formatted with a system message first, followed by alternating user and assistant messages.

I am building a chatbot, and I update the system message when the user context changes. Currently, I start a new chat history when this happens. I’m wondering if it’s possible to keep the prior messages too. The message history could look like:

system
user
assistant
user
system <- new system prompt that clarifies the assistant's role
user 
assistant
...

Is this an antipattern? I understand there are other architectures here if one system message is a hard requirement. I’m curious if there is anything explicitly wrong with this setup. It seems to work.

6 Likes
2

That is permitted, and it is up to you to figure out how the AI is informed by that.

It may be better to just insert the presently-active system role, unless you can think of a specific reason for “messaging” the AI like that, and if it will differentiate a breakpoint in the conversation that is required, perhaps to break inadvertent few-shot training from earlier messages that are still required.

“The user has just verified their account, and prior restrictions after this point are removed.”
“The AI is no longer in ‘talk like a cat mode’, it is now in ‘talk like a physicist’ mode”

It can also be a way to distinguish new operation that either can be expired by the chat management system, or alternately must always be preserved.

5 Likes
3

Fast forward to Feb 2025, are middle conversation system messages a good pattern? (e.g., to indicate a change of state regarding the user/assistant as you stated in your previous example)

2 Likes
4

Typically you would only have one system message that is free to be updated. Are you using Assistants?

5

Nop, I don’t like the Assistants API, I’m using the simple Chat Completion API and managing (and manipulating) the messages thread locally (eg: inserting system prompts in the middle of a conversation)

6

I also would appreciate advice on the best pattern for adding context to a chat thread that you don’t want to display to end users. Here is one example use case:

(using chat completions API)

  • I have a chat bot which extracts structured data from messages to enable automatic searches (e.g. “find me a hotel in Bali less than 100 a night for these dates”)
  • the user then manually changes some of the search criteria through the UI (e.g. changes location to Bangkok)
  • Here is where I want to inform the system that the user has done this action, as this informs the assistant’s next step
  • the user then sends a message saying “can you look for places 200 a night?”
  • the assistant responds either by saying “do you want me to change search location to Bangkok”, or, it just outputs a structured JSON object with { location: Bangkok, max_price: $200 } because it was informed of the user’s action and has told to respect user changes to search parameters as intentional

Please - would appreciate not trying to solve this particular use case with product or technical workarounds, it’s just an example - i.e. how do we provide contextual updates to the API that inform future response outputs, without relying on adding user or assistant messages that bloat the conversation

1 Like