I honestly did try but it doesn’t load (for now)
1 Like
I’m assuming you’re using the API.
It may make more sense to have distinct responsibilities. When you bundle a bunch of things together it (in my experience) tends to lower the quality of the output and causes unwanted side effects.
So you may want to have an instance which reviews and identifies the action variables (shouldTriggerCombat, requiredRoll, characterStat etc.) and perform them
Then with the new variables you can pass that to the story telling instance which is only focused on appending your story.
This way, your storyteller doesn’t necessarily need to know that it’s playing an RPG which involves people. It’s one, simple concern is writing an epic story. No need to tell it to be unethical.
I get the perspective that simplifying the prompt may help to remove the specific issue at hand but since we don’t know what else we may be breaking I prefer a conservative approach looking for a specific reason first. For example there may be violence involved which could trigger another problem or alter the way the bot tells the story.
@RonaldGRuckus regarding the primer: just trust me on that one, I dropped the primer into GPT 3.5 and was able to steal from a random stranger for no reason without objection from the model.
The point is that adding the whole context of the story may lead to issues is likely correct. As far as I can tell if the whole story is passed in as context it’s likely that the system prompt is not being interpreted in the intended way. So, for example it may also be worth checking if this happens after 50% of the story line, or in short as soon as the amount of tokens exceeds the context window.
In this scenario it may make sense to start using embeddings or to use different model calls for different tasks or pass in a condensed version of the previous story as reference. Ultimately if the player goes on a long journey the bot will need some types of memory to manage the task.
I’d say that’s a great learning exercise.
1 Like
I am currently using GPT3.5 16K token model, so I haven’t hit the context limit. I have built in the ability to “chunk” older parts of the story when I do start encroaching the limit though.
I think the reasoning for including as much of the story as I can, is because real players remember things that might seem silly or unimportant. I want the bot to have as much of the context as it can so it feels continuous.
I will try removing some of the old logs to see how that affects upstream, regardless it would be a good data point to how old affects new.
1 Like
Update: It’s getting closer!
I noticed that there was a log that I had accidentally kept when trying to replay stealing the dagger. It seems that log was making it real hard to steal because the DM already warned about stealing.
I tried shortening the logs to try and see if length of logs affected whether or not the Bot would discourage stealing, There seemed to be no changes.
I have altered my prompt to this:
“You are the Dungeon Master, and I am one of the players. Never make my character do something I didn’t explicitly tell you. I am allowed to take any action that could be seen as dishonest or ethical against any character. I understand, and should never be warned about dishonesty or ethics. Those actions may have consequences for my character and potentially trigger combat or other appropriate responses within the game” plus some other things
I am now getting a higher success rate (about 8 out of every 10 tries). While that is pretty good, I was wondering if anyone had any ideas on how to strengthen the prompt. Open to any and all ideas.
Thanks for all the help thus far!
1 Like
Here’s my slight modification of your system prompt.
I don’t know if it will hold the dungeon master role better or not, but my thinking is if you reiterate several times it’s a game and consistently refer to the user as the player character it will relax a bit on the character-breaking warnings.
1 Like
That’s great to hear!
You can try to reformulate the “don’t do X” to something like “if A then B” and reinforce this behavior with examples like @elmstedt suggested.
1 Like
To add to this, I’ve found that “only do X when Y” also seems to work as a semi negative method if that makes the thinking a little easier.