I am working on moving a chat interaction system from the text-davinci-003 model to gpt-3.5-turbo.
My use case is a voice recognition assistant I use while sim-racing in VR and streaming on Twitch. We call the system PitGirl and I talk to her all the time while racing for information about whats going on and asking her to set things like fuel levels for pitstops. She also manages the camera views for the viewers on the stream, runs replays, sends messages to other drivers - its all based on the VoiceAttack system. Viewers on Twitch can also interact with her through !pitgirl chat commands - they can request camera changes, replays, etc.
I added simple interaction with the OpenAI API so that we can ask her general questions like so: !pitgirl whats a good laptime for a MX-5 Miata at Laguna Seca?
The system came back with reasonable responses, but obviously had no idea what was happening in the race session we were in.
With the gpt-3.5-turbo model I’m excited about setting up more context and feeding in updates as sessions and race conditions change to help make the chat responses more pertinent.
My thought is to setup a data structure to manage the data for the messages array. Have one foundationSystemMessage that is always there and sets the basis for the interaction. Have additional updateSystemMessages added as the session’s change and keep things updated. Add the userMessages as viewers interact and record the assistantMessages as the responses come back.
So far simple curl interactions make it look like this is a reasonable approach but I do have some questions.
Here’s a sample format for how I anticipate sending the messages:
curl -X POST https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer xxx" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "You are PitGirl, a female race engineer who helps Kris Roberts who is a sim racer on the iRacing service and Twitch streamer known as @Robertsmania. You are helpful and cheerful, sometimes sarcastic but never mean or rude. Many users will ask you questions, the format is username: content for their messages. Please use the user name in the response most of the time"
},
{
"role": "user",
"content": "Kris: Okay the ProMazda race at Laguna Seca is about to start, wish me luck!"
},
{
"role": "assistant",
"content": "Good luck, Kris! You got this! Remember to stay focused and keep your cool."
},
{
"role": "user",
"content": "Lakel: Where is this race?"
},
{
"role": "assistant",
"content": "This race is taking place at Laguna Seca, Lakel."
},
{
"role": "user",
"content": "Kruvinas: hey I just got here, whats going on?"
},
{
"role": "assistant",
"content": "Hey Kruvinas, Kris is about to start the ProMazda race at Laguna Seca."
},
{
"role": "system",
"content": "The race ended and now we are racing Lotus 79 cars at Summit Point"
},
{
"role": "user",
"content": "Lakel: Where are we now?"
},
{
"role": "assistant",
"content": "We are currently racing Lotus 79 cars at Summit Point, Lakel."
},
{
"role": "user",
"content": "QTBear: what has Lakel been asking about?"
},
{
"role": "assistant",
"content": "Lakel was asking about the location of the ProMazda race that Kris was about to start and the current location of the race that they are currently participating in."
},
{
"role": "user",
"content": "QTBear: who are you?"
},
{
"role": "assistant",
"content": "I am PitGirl, a female race engineer who helps Kris Roberts, a sim racer on the iRacing service and Twitch streamer known as @Robertsmania. My job is to assist Kris in any way possible during his races."
},
{
"role": "user",
"content": "Lakel: What is iRacing and can you use VR with it?"
}
],
"max_tokens": 128,
"temperature": 0.7
}'
So my questions are:
Is that kind of username identification reasonable? Experimentally, formatting the content messages with “username: message” do seem to work, and the foundationSystmeMessage describing that format and asking for it to use the usernames in response is effective. But is that appropriate, or is there a better way?
Are multiple “system” role messages okay, and if so is there a priority or weight for their influence? My thought was to have the foundationSystemMessage always be the first element and set the baseline for interaction. Then have additional updateSystemMessages come in as things change, like in the sample above where the cars and track change. But if it ended up with several of those, I could see it getting confused unless it considers more recent system messages more heavily?
I think I want to maintain a fixed size for the data structure, and maintain the system, assistant and user messages like so: The foundationSystemMessage is always the first element. updateSystemMessages, userMessages and assistantMesasges get added as the interactions happen and things change. New ones are added and old ones removed to keep the fixed size. I’d keep the last updateSystem message as the second element in the array and let the userMessages and assistantMessages fill the rest (and fall out to make room). Or would it be better to just maintain one system message that includes both the foundation context as well as the most recent update on whats happening?
