My app allows users to access large quantities of reference books within a very specific field of study. I’ve found ChatGPT to be a very useful tool in my own use of this material and would like to extend those benefits to my users by incorporating it into my app. My first idea was to provide a small selection of structured prompts, where, for example, the user just selects “People” and I create a prompt that says “Tell me about the people mentioned in or referred to in [context]” where “context” is the book and chapter that the user is reading. This works very well, but it is very static. I often find I want to ask a follow-up question.
My concern is that if I open up the app to allow the user to simply type a question, he or she could use it to ask any question. I’d have to provide context in the form of the history of the chat, and that costs me money. Furthermore, the user could wander off into a discussion of classic cars, knitting, philosophy, or whatever — none of which is the subject matter of my app — and I would have to pay for it.
So my question for the community is, how do I enable a more open-ended chat experience in my app without paying for the user to ask irrelevant questions at my expense? How do I control the subject matter of the user’s prompts or limit the scope of the model’s responses?