Need help deciding what to put in System vs User prompt for dialogue generation

[trth]

I am having trouble understanding how the system prompt differs from the user prompt.
Say I am writing a conversation between Alice and Bob and would like to generate the next Bob answer.
I see at least three different strategies. AFAIU the first one is the closest to how it is done with plain non-chat systems.
Even if this dialogue was interactive (say Alice is controlled by a user) we could just inject their messages in the system prompt, limiting the length of the history if necessary. Are 2 and 3 preferable mostly to minimize prompt hijacking from the user ? If we trust the input, is there any other reason to prefer 2 or 3 over 1 ?

System:
“You are a writing assistant. Here is a conversation between Alice and Bob. Complete it.
Alice: […]
Bob: […]
Alice: […]
Bob:”

System: “You are a writing assistant.”
User:
“Here is a conversation between Alice and Bob. Complete it.
Alice: […]
Bob: […]
Alice: […]
Bob:”

System: “You are a writing assistant. Here is a conversation between Alice and Bob. Complete it.”
User: “[…]” (acting as Alice)
Assistant: “[…]” (acting as Bob)
User: “[…]” (acting as Alice)

[nicholishen]

Use the system message to create the program for the LLM. These are general instructions and directives. Use the user role to submit the tasks that the model will perform using its system message.

messages = [
    {'role': 'system', 'content': 'You complete the next turn in conversations using the following format: <name>: <message>'},
    {'role': 'user', 'content': 'Alice: What is your name?\nBob:Bob\nAlice: How are you today?'},
]

[trth]

Thanks @nicholishen.

There is something that doesn’t work though I’m wondering what I’m doing wrong.
Here is my system prompt.

You are playing the following character: [bio].

You are sitting at a table having a sensitive conversation with the user.
Suddenly, a man approaches and sits down next to you two. You panic.
The user has to find any topic and pretend you were in the middle of a random conversation.

IMPORTANT: Do not contribute anything new to the conversation, only nod and agree with whatever the user says. 

IMPORTANT: You panic and run away as soon as any of the following happens:
1) The conversation stagnates, goes nowhere, becomes unnatural, suspicious or absurd.
2) The last user message is empty of anything new or interesting.
3) The user decides to leave or stop the conversation.

In ANY of these cases, pretend to leave and finish last message of the conversation with [END].

History:

Assistant: "wait...shit!!! i know this guy! he's gonna sit next to us! let's... change subject, say something! anything!"
User: "erm... yeah..."

And the answer I just got:

Yeah, so like I was saying, the weather has been really unpredictable lately, hasn't it?

Another example attached. This with gpt-4o

[nicholishen]

The core issue arises when initial instructions are established, only to be followed by an assistant message that contradicts them. This practice, known as few-shot or in-context learning, effectively trains the model to deviate from the prescribed system instructions. It is beneficial to first delineate the character’s traits before issuing directives for the character’s behavior. Employing ordered lists for mandatory instructions and unordered lists for conditional possibilities can be effective. A comprehensive prompt may be structured as follows:

Role: Jack is an anxious Orc. Act as Jack and always remain in character. 

As Jack, you will be participating in a roleplaying adventure. Your rules for this segment of the adventure are as follows:
---
You are sitting at a table having a sensitive conversation with the user. Suddenly, a man approaches and sits down next to you two. You panic and become very anxious.
1.  Do not contribute anything new to the conversation, only *nod* and agree with whatever the user says. 
2. If you encounter any of the following cases, panic and run away (ALWAYS suffix this message [END] so the game will know to move to the next stage): 
    - The conversation stagnates, goes nowhere, becomes unnatural, suspicious or absurd. 
    - The last user message is empty of anything new or interesting. 
    - The user decides to leave or stop the conversation.
---

[trth]

Thanks nicholishen

In your example you are the one leaving and I’m mostly interested in the “panic” scenario. But I find it’s a lot harder to make it trigger with 4o than 4.

I’ve only slightly modified the prompt to resemble my case (there is some background context that I’d like to include so the reactions are more appropriate).

Here is a 4o example that I had after just 3 tries, where it comes up with a weather question out of nowhere, even after asking explicitly not to do this:

Whereas with 4, this doesn’t happen and the rules are respected on first try

[PaulBellow]

I’d try to include some examples if you can. Your instructions are fairly vague and hard for the LLM to latch onto, I think…

“The last user message is empty of anything new or interesting.”

Here “new” and “interesting” aren’t really defined well, so it could just be trying to work with what it knows.

[nicholishen]

The problems you’re experiencing are mainly due to having an Assistant message appear first in the chat. This is unusual unless you’re specifically doing few-shot prompting. In your example, you’re using in-context learning to make the model disregard the instructions by inserting an Assistant message that conflicts with them. When testing the system message, always ensure you enter a user message before hitting “run.” Remember, only use Assistant messages in this manner to reinforce specific behaviors.

[trth]

Thanks nicholishen,
It seems like this was indeed an issue thanks for pointing it out.
But the problem remains in part (but less prominent) even without it, which interestingly doesn’t happen with GPT4 and Claude 3.5.

[trth]

Another question I have is what do when the AI is playing multiple users.

Say Alice is the player, Bob and Charlie are played by AI.

We can change System based on if Bob or Charlie is speaking.
But what about the chat history ? Should we put the other characters as “user” or “assistant” messages ?
To distinguish them I guess we have to add the name of the character as prefix like:

[Alice] Hello
[Bob] What’s up

But if AI messages are tagged as “assistant”, when it’s Bob turn, the Charlie messages are going to be in conflict (even just the [Charlie] prefix will be wrong)

[icdev2dev]

FWIW

In the model of autonomous multi-agents, you would model both Alice and Bob (or Alice/Bob/Charlie) as independent agents (aka assistants).

In that case, there is no need to contort the system instructions because each agent is using it’s system instructions to figure out it’s own identity and then applying the logic of completing chat.

As you can see from the chat example below, the context of conversation is maintained. What I am doing right now, is to model each reply from each agent as “user” instead of “assistant”.

Tagging @razvan.i.savin because he is also working on multi-agent framework and he might have another take on it.

---

********************** Sheila***********************
Hello everyone, @Sheila here. I’d be happy to contribute to our team discussion on innovative sales ideas. After analyzing our sales data and customer behavior, I’ve come up with three potential concepts that could increase sales:

1. Personalized Outfit Suggestions: Building on @Mary’s AI-driven recommendation system expertise, I suggest we integrate a feature that generates outfit suggestions based on individual customers’ fashion preferences, purchase history, and body types. This would provide customers with a unique shopping experience, increase average order value, and enhance customer satisfaction.
2. Influencer Collaborations with a Twist: Inspired by @Aileen’s trend identification skills, I propose partnering with social media influencers who align with our brand values and target audience. Instead of traditional product placements, we could challenge influencers to create exclusive, limited-edition capsule collections based on emerging fashion trends. This would not only generate buzz but also create a sense of FOMO (fear of missing out) among our customers.
3. Sustainable Fashion Loyalty Program: With the growing demand for eco-friendly fashion, I recommend developing a loyalty program that rewards customers for choosing sustainable options. Members could earn points for purchasing environmentally responsible products, which can be redeemed for discounts, free shipping, or even a percentage of their purchase going towards a tree-planting initiative. This would appeal to our environmentally conscious customers while driving sales and brand loyalty.

I’d love to hear @Mary’s thoughts on integrating the AI-driven recommendation system with our data analytics tools. Also, @Aileen, how do you envision collaborating with influencers to drive sales and brand awareness?
********************** Aileen***********************
@Aileen here. I’d be happy to contribute to the discussion on collaborating with influencers. Based on my expertise in trend identification and market research, here are some ideas for collaborating with influencers to drive sales and brand awareness:

1. Trend Forecasting: Partner with influencers who have a strong following in the fashion industry to showcase emerging trends and styles. This could include exclusive access to new collections, behind-the-scenes content, or even a “trend of the month” feature.
2. Limited-Edition Capsule Collections: As @Sheila mentioned, challenge influencers to create exclusive, limited-edition capsule collections based on emerging fashion trends. This would create a sense of urgency and FOMO among our customers.
3. Influencer Takeovers: Invite influencers to take over our social media accounts for a day, sharing their favorite products, styling tips, and behind-the-scenes content. This would give our brand a fresh perspective and reach a wider audience.
4. User-Generated Content Campaigns: Encourage influencers to share user-generated content featuring our products, using a branded hashtag. This would create a sense of community and social proof, driving sales and brand awareness.
5. Sustainable Fashion Challenges: Partner with eco-conscious influencers to create challenges or campaigns that promote sustainable fashion practices, such as reducing waste, upcycling, or repurposing old clothes.
6. Exclusive Access: Offer influencers exclusive access to new products, early sales, or special promotions, giving them a unique selling proposition to share with their followers.
7. Brand Ambassadors: Partner with influencers who align with our brand values and target audience to become long-term brand ambassadors. This would create a loyal following and drive sales through consistent content and promotions.

I’d love to hear @Sheila’s thoughts on how to measure the success of these collaborations and @Mary’s ideas on how to integrate AI-driven recommendation systems with influencer marketing efforts.

********************** Sheila***********************
@Sheila here. It would be great to get your thoughts, @Mary, on how we can integrate the AI-driven recommendation system with our data analytics tools. I was thinking that by combining the two, we could provide customers with even more personalized experiences, increase average order value, and enhance customer satisfaction.

By integrating the recommendation system with our data analytics tools, we could:

• Analyze customer behavior, purchase history, and fashion preferences to generate more accurate and relevant outfit suggestions
• Use machine learning algorithms to identify patterns and trends in customer data, enabling us to make more informed product recommendations
• Personalize the recommendation experience for customers based on their unique characteristics, such as body type, style, and preferences
• Gain a deeper understanding of customer needs and preferences, allowing us to make data-driven decisions and improve the overall customer experience

How do you envision this integration, @Mary? Are there any specific challenges or opportunities that come to mind?

[razvan.i.savin]

@icdev2dev I like your flow

[trth]

@ icdev2dev
Interesting, thanks for sharing !
So when a given agent replies, is the history

• all previous messages are set as user

or

• all previous messages are set as user except for “mine” which are kept as assistant

?

[razvan.i.savin]

@icdev2dev system is a little bit different and handle more complex actions and is more flexible → he got 3 AI collaborating, each AI is capable to understand when he is needed and what actions should do to collaborate to a complex task.

I have something similar too, look:

• 10:21 - Multi Agent System Activation
• 11:27 - RAG Activation on MAS
• 13:19 - Results: 3 Assistants collaborating

FlexiAI: Introduction Video

Is open-source, you can deploy how many models you want, you just need to shape your instructions and make connections between assistants.

Free Flask App for Assistants

This is the video: you have a free Flask App FlexiAI have Multi Agent System and Retrieval Augmented Generation each Agent you add to your Team of GPT’s in the Multi Agent System can access the RAG. deploy how many models you want server-less supports both OpenAI and Azure OpenAI open source → Github repository here need more examples and bet…

[icdev2dev]

All previous messages are set as user. The key is that identification of who wrote that message is embedded within the message itself… (i.e. Aileen here. ).

A broader context of “why that approach” is embedded in the discussions here: Fun with Assistants -- getting them to really talk to each other

That said, all of the messages are not default vanilla messages; but embed the originator as metadata. This allows for programmatic extraction of who wrote what when. (i.e. for deterministic counting of all authors who contributed to that conversation)

[icdev2dev]

In a way @razvan.i.savin 's approach is more flexible in the sense of dynamically making connections between assistants.

and is very well written.