Question around the structure of system, user, assistant

API
1

Hi,

I’m trying to make a discord bot that responds from time to time to what’s being written in the channel. In the past i used to just keep track of the entire conversation in a list and then pass the entire list to the model and say : this is what was said, what’s your reply?. I’ve seen that the API has evolved and we now need to pass a list of key value pairs.

What has been recommended is:

System
User
Assistant
User
Assistant
User
Assistant

Considering that I want to bot to answer just from time to time, would this work?

System
User
User
User
User
System (i tell the bot to adapt it’s tone based off what was said above)
Assistant
User
User
System (tone change again)
Assistant

Or does it really have to be system, user, assistant, user, assistant at all times? I can basically group what everyone is saying and put it in a single user statement.

My issue is that i’m trying to give it a memory and have it say relevant things and keep track of the conversation but it’s failing miserably. I need to add that I have also fine tuned the model based off what was said in discord and i think the fine tuning might also have something to do with the quality of the responses.

2

You can add system messages at any point, but you can also give the instruction of tone change as a user message.

The model behind the chat completion API cares only a bit about the message type.

Finally, you can also name the users, so that the chat api has a little more information, and you can take this from the user names in discord.
image

So, in short. Yes, your model would work, but you don’t need those messages to be system, and you can name all users for more context.

ref: https://platform.openai.com/docs/api-reference/chat/create

3

Thanks a lot! So to reiterate, i can make a list with system, user,user,user,user , pass this to the model, get the reply and then add it to my list to become system, user,user,user,user,assistant , then add a few more user comments and send the whole thing system,user,user,user,assistant,user,user to the model and it should be able to make sense of this whole thing.

4

Remember whatever you send as user or assistant the LLM will interpret as a SINGLE user conversation between it and you. So what you’re doing may be easier than you think. You can probably just say in the prompt like this:

[prompt]
Here’s a social media conversation:

bob: Hi
alice: Hi Bob
bob: I hate argumentative people.
alice: no you do not!
bob: yes I do

What would alice say next?
[/prompt]

and by including the ENTIRE conversation into a single “user” prompt you’d say to the GPT. What is a funny reply to the above conversation form Joe. Or “Write a joke involving the conversation above” etc.

So you don’t need to build a big list like you were doing. It’s just a single question to the AI to get your new reply from the bot.

1 Like