What is the difference between putting the AI personality in system content and in user content?

Prompting
, , , ,
1

For example, at these code:

task = "Help me summarize the article abstract I gave along with keywords"
text = "In this paper, ....  "
content1 = [{"role": "system", "content": task},{"role": "user", "content": text}]
content2 = [{"role": "system", "content": ""},{"role": "user", "content": task + "\n###\n" + text + "\n###"}]

Do these two writing methods have the same effect?

Or will gpt give more weight to focus on my task by writing it in system content, like what I wrote for content1?

What is the mechanism of gpt’s focus on my task?

2 Likes
2

Welcome to our developer community.

What have your tests shown?

As far as I know at the moment, the user/assistant have more weight than the system message, but that’s not guaranteed to always be the case as ChatML is changing/evolving/improving.

I would try both maybe a half dozen times and compare/contrast to see the difference, keeping the other parameters the same.

You can also try adding it this way

SYSTEM MESSAGE

USER: MAde up user example
ASSISTANT: Made up assistant example
USER: The user’s input

etc, etc. Experimentation is useful for tools this new.

1 Like
3

Thank you for your reply

I find that when I enter a particular character personality into system, as the conversation gets longer and longer, gpt becomes more oblivious to the charactor I give in system.

I can’t quite understand the significance of ‘system’ role being played in my conversation, maybe I haven’t used it right?

1 Like
4

I remember a thread from someone who was trying to keep the assistant in character, and how they got better results using davinci than gpt-3.5-turbo. They later realized that message order apparently mattered. If the personality was always appended in a system message as the very last message, it would mostly adhere to it. Maybe that helps.

There’s the thread:

2 Likes
5

Also 3.5 is much worse at following system message
4 is much better at it
From experience

image
image1250×1504 131 KB

From: OpenAI API

Interesting idea about putting system as last, I might have to test that out.

I really wish we had something cheap/fast like 3.5 that would follow the system message like 4 does

1 Like
6

Although the user message is “heavier” than the system message, it’s good practice to use the system message for a couple reasons

  • Reverse engineering
    – A user can retrieve your database and understand your process easily by simply asking what they just said. If it’s sent through the system message, GPT is more relutant to admit that it even exists

  • Good Practice
    – As mentioned, they are continuously training GPT to give more power to the system messages. It’s better to adapt to a “weaker” version, and make it work. Knowing that it will continuously get better. Rather than focus on “hacking” it in the user role and deal with progressively worse results.

  • Modularity
    – It’s much easier to navigate and manipulate the conversation with business logic when your injected messages can be identified by their system role. Rather than bundled in with the user role

I don’t know how to categorize the last one, or have any proof, but GPT recognizes system messages as truthful logs, and insights. Considering that we are usually roleplaying with GPT to a certain extent, having a natural, structured, non-hacky roles (imo) must benefit the cohesiveness of the conversation.

I have never had any issues with using the system message.

3 Likes
7

Great! that’s exactly what I was looking for

Seems there is already a dedicated thread on this topic, then I will move over there to discuss it.

Thanks guys!

8

is ？What is best way to use “System” field in “Mode: ChatBeta”?

9

thank you for this actionable response. Honestly I read this and a lightbulb went off