Question about prompting - Instruction help

Please visit this playground preset:

So the setting is, pretend you’re a University advisor. And underneath is a list of CS courses.

I gave it a few extra instructions:

  • Only reply with information from the provided dictionary.
  • If information isn’t found, reply “not found”
  • Only recommend courses based on prerequisites, do not recommend classes based on course title or course content.

So the problem I keep having is with this made-up user prompt,
“What are the classes I need to take in order to take CS 3801 if I’m in CS 1010 right now?”

From the list,
CS 3801’s prereq is CS 2013
CS 2013’s prereq is CS 2012
CS 2012’s prereq is CS 2011
CS 2011’s prereq is CS 2010
CS 2010’s prereq is CS 1010

However, every now and then, the model would shove CS 2148 as a required course. So then I asked, “Why do I need to take CS 2148 before CS 3801?”

And the model would reply because in CS 2148, the students would learn essential knowledge in order for them to understand CS 3801.

But in the instruction, I clearly stated that do not recommend classes based on course title or knowledge learned from course. I couldn’t get the model to just follow the prereq only.

Please help!


Personal notes on the example:

  1. (The most important) - you’re trying to solve a multi step problem in one shot using generative models… Most likely you’ll never get viable solution for commercial use.

Split your recommendation process into several simple steps and you’ll get something flexible, reliable and easy to work with/maintain.

  1. Personally I use following outline for these type of prompts:
  • general setup (you’re a skillful…)

  • task

  • data (if that many needs to go in system message, otherwise send as first user message)

  • task summary again (when long data)

  • rules
    – 1. As a numbered list

  • expected output

  • formatted as following: expected output format example

  1. Sending user message “as is” to get a reply from assistant in these types of apps is never a good idea. Something like:

Based on the context data below:

Your contexts

Answer the following questions from the user:

“User inquiry goes here”

  1. But even the answer process in these types of apps is not a single shot. You need to nap the flow of how you want your answer and how you validate the AI answer before spring it to your user…

Thank you for your help! I am trying out different techniques you mentioned hoping to get better response!