How message history inside the system message influence the gpt-4o model?

Prompting
1

Hello guys. Recently I started to work on a project where I decided that, for a better controll of where in the prompt I put my message history, I will interpolate them in the system prompt.
So now I have something like:

System prompt instructions…

Conversation history: {messages} -#here it will be the array of Assistant message, HumanMessage, etc.

User last message: {user_message}

And I take this prompt and send it to the API as a SystemMessage

So the question is if this approach will influence the behavior of the model because I send everything in a SystemMessage?
I tried to test a little bit with the other approach where I create a SystemMessage followed by all the previous Messages and the last User message.

Is there any differences between those in terms of how the LLM will interpret them?

2

This seems like a strange implementation. Why are you placing the history in the system message rather than sending the actual history as is?

I imagine this would negatively affect the model’s accuracy, because OpenAI doesn’t have a reason to train models in this format. If you really want to know, though, you’ll need to experiment yourself and determine what the impact is for your use case.

3

Thanks i will look into it.
The reason for implementing it like this was just to play around. I want to understand how the ChatPromptTemplate is transformed and infered on the open ai side.
In my head I was thinking of 2 possible ways to do it(forgive if i am going to say a stupid thing i am new to this):

  1. That array of SystemMessage, AiMessage, HumanMessage is converted into a big chunk of text and infered. And in this scenario i was thinking that if i put a SystemMessage with the history inside it will make no diference
  2. That array will not be transformed in a chunk of text and the array will be embedded and sent as it is to the to model. In this case i assume that yes, the fact that i have just an array with a SystemMessage with all the info inside it will make a difference in how the embeddings are generated.

So this was my curiosity. And sorry again if I put stupid questions i am just trying to understand better how everything works :slight_smile:

4

System messages are primarily meant for instructions. You can add “history” as examples for guiding a desired output.

Basically, the difference is that in case of conflicts, system messages have higher relevance in steering the model’s behavior.

Since you are exploring, you might want to have a look into the model specs, where you can read more about how it works in details.

2 Likes
5

Thank you guys.
This document helped me find the info about how each message is passed to the model. I will leave this here in case of someone needs it:
" A message is converted into a sequence of tokens before being passed into the multimodal language model, with the fields appearing in the order they are listed above. For example, a message with the fields

{
    "role": "assistant",
    "recipient": "python",
    "content": "import this",
    "end_turn": true,
}

might appear as

<|start|>assistant<|recipient|>python<|content|>import this<|end_turn|>

where <|...|> denotes a special token."

1 Like