My app allows users to access large quantities of reference books within a very specific field of study. I’ve found ChatGPT to be a very useful tool in my own use of this material and would like to extend those benefits to my users by incorporating it into my app. My first idea was to provide a small selection of structured prompts, where, for example, the user just selects “People” and I create a prompt that says “Tell me about the people mentioned in or referred to in [context]” where “context” is the book and chapter that the user is reading. This works very well, but it is very static. I often find I want to ask a follow-up question.
My concern is that if I open up the app to allow the user to simply type a question, he or she could use it to ask any question. I’d have to provide context in the form of the history of the chat, and that costs me money. Furthermore, the user could wander off into a discussion of classic cars, knitting, philosophy, or whatever — none of which is the subject matter of my app — and I would have to pay for it.
So my question for the community is, how do I enable a more open-ended chat experience in my app without paying for the user to ask irrelevant questions at my expense? How do I control the subject matter of the user’s prompts or limit the scope of the model’s responses?
When you send the user question to ChatGPT, you can tell ChatGPT to limit its response to certain topics in the system message. For example, you can say “If the user’s question is not directly related to the given book, politely reject it.”.
I’ve been having fun today telling ChatGPT to only answer questions related to the specific material, then tricking it into talking about something else. It profusely apologizes, of course, but then if I ask it to give me some ideas for a prompt that would keep it from violating my rules, it will do so — even though doing so violates my rules.
I’ve found that it helps to repeat the rule at the beginning of each new question. I might be able to come up with a brief form of the rule and prepend it to each user question. I was hoping to present the rule as the first item in the conversation and not have to repeat it.
You need to craft a good system prompt to restrict the AI to discuss topics within certain limits only. However, I think providing a bit of the history of the chat is inevitable to make the flow of conversation better. You don’t have to include everything, maybe just the last 5 pairs (user + assistant) or so. It will be frustrating for the user, from my own experience, when the AI does not know what I was talking about from previous conversation.
