What is the best way to pass user related facts to chat completions in multiuser conversations

Hi there. At the moment I’m experimenting with adding bot (which is just a gate to openai api) to small group chats (10-15 members).
I have a database which have some facts about each user (e.g. age, name, some funny facts and maybe his relation to other chat members).
Also if bot is participating in thread discussions I’m passing the whole thread to it on each turn (chatgpt is able to see the whole thread along with user’s names.
Overall, I’m simply calling

       response = await openai.chat.completions.create(
            model="gpt-4o-mini",
            messages=chat_messages
        )

where chat_messages look like this:

[
   {
      "role":"system",
      "content":"Your are the bot in chat, you're playing the role of a real person."
   },
   {
      "role":"system",
      "content":"\nCurrently you're answering to user 999888999\nHis name is John Smith\nAdditional facts about him: he's 32, he likes playing Skyrim"
   },
   {
      "name":"999888999",
      "role":"system",
      "content":"My name is John Smith\nAdditional facts about me: I'm 32, and I live in Boston"
   },
   {
      "name":"777666777",
      "role":"system",
      "content":"My name is Jane Doe\nAdditional facts about me: I'm 25, and I live in Seatle"
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"@bot Hi, how are you? What is my name?"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Whats up John? If you have something to share - tell me"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"I forgot who am I and whats my name, tell me please"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"You are John, we both now you! Are you ok? You can share if any thing bothers you"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"Where do I live?"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"You live in Boston"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"An what about me?"
         }
      ],
      "name":"777666777"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Jane, you live in Seatle. How it is there?"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"I bought a car and now I live in car in Seatle"
         }
      ],
      "name":"777666777"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Wow! Gratz Jane! Hope you feel comfort. Which car did you get?"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"It's Jeep"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Jeep is great car, you can drive offroad. What plans do you have with it?"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"By the way, who bought it?"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Probably it was you John. It's your new car!"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"Who told you before that got a Jeep?"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"You did, John! Were you joking?"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"Maybe it was Jane?"
         }
      ],
      "name":"999888999"
   },
   {
      "role":"assistant",
      "content":[
         {
            "type":"text",
            "text":"Oh, yeah, right! She told that she bought a car and know live in it. We should check it with here. Anyway that's great someone here has a car!"
         }
      ]
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"Yeah, your are right"
         }
      ],
      "name":"999888999"
   }
]

So I am using userid as a name in completions call (there might be several users with same name, so I want them to be distinguished).
And I also passing system messages with name of specified users who is participating in the conversation giving chatgpt facts about these users.
It works, it definately gets the facts, even if first user is asking about the second user in conversation.
But it feels that “name” is not working perfectly at all, and bot get’s easiely confused. As in conversation before - bot didn’t recognised correctly who bought a car. Bot asked Jane what car did she buy, but John answered “Jeep” and bot started to think John bought a car despite he didn’t tell it.

It gets even more frustrating after I’ve added “memory” (bot wrote new facts about users to database) - it just messes the facts and add fact of different users since for some reason it can’t correctly recognise the previous part of passed converstation.

So, what is correct way to pass facts and names to chat completions so chatgpt won’t get confused between several users in conversation?

I’ve also tryied to pass facts from the ‘first person’ in the beginning of the thread like so:

[
   {
      "role":"system",
      "content":"Your are the bot in chat, you're playing the role of a real person."
   },
   {
      "role":"system",
      "content":"\nCurrently you're answering to user 999888999\nHis name is John Smith\nAdditional facts about him: he's 32, he likes playing Skyrim"
   },
   {
      "name":"999888999",
      "role":"user",
      "content":"My name is John Smith\nAdditional facts about me: I'm 32, and I live in Boston"
   },
   {
      "name":"777666777",
      "role":"user",
      "content":"My name is Jane Doe\nAdditional facts about me: I'm 25, and I live in Seatle"
   },
   {
      "role":"user",
      "content":[
         {
            "type":"text",
            "text":"@bot Hi, how are you? What is my name?"
         }
      ],
      "name":"999888999"
   }

However is seems to work same way. Sometimes bot perfectly distinguishes each user and their messages in long thread (I’ve asked bot to summarise the thread and he did almost perfectly) and sometimes it just messes everything.

I realize the 4o-mini is simpliest model, however when directly passing “name” I expect it to correctly differs the users.

Maybe I’m doing something? Would much appreciate is someone helps to find out.

And quick follow-up question: can someone clarify main difference between role: system and role: user? I mean docs give some info about that, but I tried passing facts about users to system, and sometimes 4o-mini refers to them as “ said that…” yet username said nothing, the fact was passed as role: system (even without name, just role:system + content). I thought system is kind of “inside” data which cannot be exposed as “user said that”

You should pass the group chat members and their pecularities in the “system” context.

“You are participating in a group chat. The group members are :
** Jane Doe ** :
Additional facts: I’m 25, and I live in Seatle
** Jack ** :
…”

Then when you compose the message from individual users, you would do some thing like :
“Hi there! This is Jane. I am going to purchase a car. Need some help”

That seems to be the concept I would use (and really used) if there were no “name” field in openai request payload.
I really tried that approach you described, but if using user_ids - sometimes model starts to refer to user by his user_id, and that not what I want.
On the other hand if I give up using user_ids there can be two guys named John (there are not somuch people using surnames in messenger) and I’ll get their facts “mixed” together.

There is not so much details on how to use “name” in open ai’s Chat Completions docs.

P.S. At the moment I’m testing same conversation over and over again using different models and different system prompts.
At the moment I think I did everything described in topic correctly except for using 4o-mini. as far as I understand the root cause is that that model can be easily fooled and make some logic errors a lot.

Ok, based on lots of tests I’ll sum up my findings:

  1. Passing facts as following using role system + name works ok:
 {
      "name":"999888999",
      "role":"system",
      "content":"My name is John Smith\nAdditional facts about me: I'm 32, and I live in Boston"
   }

All further ‘real’ messages are passed in such manner (role user + name):
{
“name”:“999888999”,
“role”:“user”,
“content”:“My name is John Smith\nAdditional facts about me: I’m 32, and I live in Boston”
}

Yet the model may still may refer to them as “user said earlier”, even gpt-4o do.

  1. Mentioned mistake (where model 4o-mini was fooled by John and thought he bought a car while really Jane did) was related to gpt-4o-mini only, gpt-4o worked correctly in such case.

  2. I also had an issue with fact duplication (model decided to add a fact about user yet that fact was already known and exposed to model). I tried fixing it by changing system prompt with no luck. Workarounded this issue by getting old_fact, combining it with new fact, and then rewriting this combined fact by model to avoid repeated facts.