Using GPT for text based adventure game/rpg game master

I’ve been messing around with this for quite some time and it is very promising. I’ve tried a few interaction flows and I can’t get the characteristics I need. I’ve come up with the following design requirements:

  • It needs challenge - This is done thourhgt weighted randomized outcomes that are fed in to the conversation.
  • it needs structure - Create adventures, with milestones that gpt nudges users towards
  • It needs consequences - User actions and, more importantly, gpt-responses should update things that are not part of the conversational context, such as
    • Player health, getting/loosing items, reaching milestones, gaining XP, adding non player characters that alernate taking actions with the players.

I’ve identified a few different methods that I can use to achieve various aspects of this:
System prompt: Works quite well, specially if it’s ammended at the end of the conversation, but It get’s quite extensive and gives me very little control or ability to fine-tune.
Tools: Works really well for skill-checks and randomizing players success at things, but GPT seems to only trigger them based on the user input, never the response. I’ve been thinking of faking the response as a new player input but it feels like a hack so haven’t tried it yet.
Getting responses as JSON, for example both a text summary and what players loose what health and items. This is promising in theory but GPT doesn’t care about my schema.

{
	summary: 'Kaelum was attacked by a goblin with a bow',
	damage: [
		Player: 'Kaelum',
		amount: '2'
	]
}

Embeddings I haven’t tried this yet, but using embeddings to amend the user prompts with both rules and world knowledge in natural language would allow for a flexible system where many people could add content and make their own adventures. You could also add a lot of specific rules and game mechanics that are included as needed. For rules that have hard mechanics the embedding calls could also run code that return the outcome to the user-prompt. This still doesn’t solve the problem that the repsonse from the AI needs to trigger functions that update user health, inventory etc.

After my poor experience with the JSON responses, I was hoping to get some input from people with more experience on a good design of this type of interaction. Any advice is much appriciated!