Tagging responses with a specific phrase

Prompting
1

I’m trying to create a murder mystery game where the player can interact with the suspects, witnesses and other police.

For my suspects I have a list of “evidence points” that I want to check for, I’m doing this by sending the chat response to the completion endpoint with the following prompt

"Does this statement include any of following information? Do not repeat the information, only a yes or no answer to each option and do not provide any other options

Statement by

  1. <evidence point 1>?
  2. <evidence point 2>?"… etc

It took a bit of trial and error on the evidence points (it had some issues distinguishing between a daughter and a niece) but it now works and gives me a list of Yes, No responses.

That works fine but obviously its an extra api call and takes extra time (I have some plans to remove evidence points once the player has discovered them) which should tone it down a bit.

What I saw in a video tutorial was asking the chatbot to add a keyword at the end of their statement, I thought that would be a great option and I tried implementing it for my police assistant. I did get it to work, although I removed all the game context just to keep the prompt short - I’ve found it doesn’t work so well when the system prompt gets too long. It only worked with the specific phrase used in the tutorial though.

Eg. this is my prompt
"This is a murder mystery game. Act as the police constable assisting the detective. Reply to questions in the first person, incorporating your personality and knowledge. Do not mention that you are an AI and do not break character. If a question is outside your knowledge of the case, respond that you do not know

if my reply indicates that I want to end the conversation, finish your sentence with the phrase END_CONVO
if my reply indicates that I want to obtain a search warrant, finish your sentence with the phrase SEARCH_WARRANT
if my reply indicates that I want to do a background check, finish your sentence with the phrase BACKGROUND_CHECK"

If I say goodbye, it does finish it off with END_CONVO
If I ask for a search warrant, it asks for the reason and doesn’t include the phrase. If I ask for a background check it tells me its an AI language model and doesn’t have access to that information :frowning:

I’m open to other approaches but if your answer is embeddings, please point me at a tutorial, ideally for 5 year olds, that explains how a bunch of numbers is supposed to tell me if my player is asking for a search warrant! Because when I read about them, I thought embeddings were the answer but I guess its beyond me

2

I’ve made some progress after watching the DeepLearning Course. It’s correctly tagging the data and giving an appropriate response, however, it isn’t returning the data in JSON as requested. Its using the keys and filling them with the right data but its not JSON and if there is no action it just gives the response.

This is my current prompt

This is a game. You are a police constable talking to the lead detective on a murder case. If you are asked something outside your knowledge of the case, respond to say you do not know. If you are asked something that it outside the context of a police investigation or the parameters of the game, ask for clarification. You are able to perform any tasks a policeman can. The game engine will handle the actual specifics of the player requests.

Perform the following tasks

  1. analyse the user input for the following requests and respond with the associated keyword given in the format of :, if none of the actions are found respond with an empty action element
  • Search Warrant:SEARCH_WARRANT
  • Background Check:BACKGROUND_CHECK
  • Fingerprints: FINGERPRINTS
  • Forensic Test: FORENSICS
  1. construct a response to the detective in keeping with your character as a police constable
  2. Format your response as JSON using the following format {“action”:string,“reply”:string}

(I’ve tried rewriting that last line in many different ways but it doesn’t seem to make a difference)
I get this kind of response:
Action: BACKGROUND_CHECK\nReply: Sure, I can run a background check on Lizzy Eden. Can you provide me with any additional information on her such as her date of birth or social security number?

So I can work with that but I’d feel more confident if the response was provided as JSON

3

First – In my experience, talking about the user input as “my reply” can confuse the model, because the model is trained as the one doing the reply.
Thus, when you’re talking about user input, use another word like “my input,” “my question,” or “my statement.”

Second: The model is pretty bad at JSON. It is, however, pretty reasonable with YAML, and awesome at Markdown (especially if you provide an example/template.)
So, if you can ask it to formulate the response as markdown, I expect that to work best.
If you can’t, then YAML (with a template/example) is probably the second best option.

4

I thought the “my reply” stuff was weird and confusing too but it worked in the example I stole it from and my own attempts hadn’t gone well so I copied it word for word. I’ve moved on from that now and have got it correctly tagging the info. Its just getting it back in something that’s nicely parsable so I can separate the response to the user from the tag for the game engine. Markdown is more of a formatting thing isn’t it?
I’m not sure the YAML would actually be that different than what its already doing. I asked for it in YAML and it didn’t look any different and it still only did it when the user request was for one of the specified actions