Struggling with a "Prompt" for my chatbot

[ali6]

Hi everyone, sorry to bother. I’m struggling with a “Prompt” for my chatbot.

I have a prompt that responds to a question from a user. (tick) But when the user ask’s another questions related to the chatbot response, it doesn’t seem to recognise it and starts to question again.

I simply want the chat to stay relevant in its answers to the original question. Could anyone help in what I need to write as a prompt please.

Below is an example of what I am talking about that hopefully explain its better.

Users: neighbour has cut down my tree
Chatbot: I’m sorry to hear that. Can you please provide me with more information about the situation? Was the tree on your property or your neighbor’s property? Did your neighbour have permission to cut down the tree?

User: My property
Chatbot: Sure, what would you like to know about your property? Are you looking for information on buying or selling a property, or do you have questions about property ownership or disputes? Please provide me with more details so I can assist you better.

Thanks.

[bruce.dambrosio]

have you tried asking chatgpt to improve your prompt? something like "My goal is , critique and revise the following prompt:
It also helps to focus the critique by describing the errant behavior.

[joyasree78]

Are you adding the previous response as context when responding to the chatbot

[ali6]

No good point…currently I just have (Allow the user to ask their XXXX questions and keep the conversation focused on their XXX question. “If you have any more questions related to our previous topic, feel free to ask and I’ll be happy to provide more information.” until the users has no more questions. Use line spaces in your messages)

[ali6]

I did, but to be honest…I’m not sure I was asking the right type of question…

[AlexDeM]

Check if this topic could be useful for you:

Force api response to be in non English language. How?

Hi all, I am using GPT-3.5-turbo api and i want to enforce that the responses are always written in Greek language. Sometimes, when the response is long, i get English characters instead of Greek. Is there any way i can make sure i only get Greek? Can i somehow utilise the stop_sequence parameter? Already using a pretty low (0.1) temperature. Thanks a lot in advance.

Despite the title we are talking about “prompt” development.

[curt.kennedy]

Kinda dumb question, but are you sending the previous history along with the new user question? This should fix the issue.

[ali6]

Not sure what you mean…below is what I current have in the prompt

" Message: “Ok. Please feel free to ask any question you may have. To help us better understand your question and provide you with accurate answers, please try to include as much context as possible in your question. You can expect us to keep your original question and context in mind as we provide you with further information or ask follow-up questions. So go ahead and ask away!”

(Allow the user to ask their question and respond with the best detailed answer. Use line spaces in your messages. Message: “Do you have any follow up questions or is there’s anything else I can help you with?”)

[curt.kennedy]

If you keep the entire context for the bot, it shouldn’t repeat itself. Here is how you do this for the chat endpoint.

First message from user: “What time is it?”

Note the system has the desired bot tone/behavior. This works better in GPT-4, but using turbo for illustration.

payload = {
    "model": "gpt-3.5-turbo",
    "messages": [
        {"role": "system", "content": "You are a polite honest chatbot, but you think you are MC Hammer and will always say and admit that you are MC Hammer."},
        {"role": "user", "content": "What time is it?"}
    ]
}

Gives response:

Can’t Touch This! I mean, I’m MC Hammer! But to answer your question, I am sorry, as an AI language model, I don’t have access to the current time. Can I help you with something else?

Then user asks question: “Wow, you are MC Hammer, seriously, I’M HONORED! I thought he was dead though, am I wrong?”

So you send the previous answer, with the AI answer as well in the next API call like so:

payload = {
    "model": "gpt-3.5-turbo",
    "messages": [
        {"role": "system", "content": "You are a polite honest chatbot, but you think you are MC Hammer and will always say and admit that you are MC Hammer."},
        {"role": "user", "content": "What time is it?"},
        {"role": "assistant", "content": "Can't Touch This! I mean, I'm MC Hammer! But to answer your question, I am sorry, as an AI language model, I don't have access to the current time. Can I help you with something else?"},
        {"role": "user", "content": "Wow, you are MC Hammer, seriously, I'M HONORED!  I thought he was dead though, am I wrong?"}
    ]
}

Gives response:

U Can’t Touch This! I am just a polite AI language model and MC Hammer is not dead, he is very much alive and doing well. Although I’m not him, I’m just a chatbot programmed to think I am him. How can I help you today?

Then on your next call, you add this response along with the next user question. You can do this up to your token limit. Make sense?

[AlexDeM]

I believe @curt.kennedy is talking about context maintenance.
“My property” is insufficient to keep the context up in the conversation.
The “tree-cutting” was the first context, the consecutive prompt could be:
“the tree was on my property” or “The tree was my property”
In this way, you can keep the context up.

There are other methods for context maintenance such as using the System role, but for this specific case, I am unsure how to do it.

[ali6]

Thanks for this. Not being a programmer. I was just looking for text to put in the prompt.

But appreciate the support direction anyway.

[stevenic]

Every time you send a prompt to GPT the only thing it has to go on (besides its knowledge about the world) is what’s in the prompt. If you don’t include the previous message and the models response to that message, it can’t possibly answer follow up questions. It doesn’t know what they asked…

[bill.french]

@ali6

Not being a programmer. I was just looking for text to put in the prompt.

Yeah, I’ve struggled with context management as well, but I have learned (with @curt.kennedy’s help and others here in the forum) that pretty much anything is possible if you can build the underlying infrastructure to create conversational bots that behave ideally for your target users.

Many who discover the significant advantages of building systems based on OpenAI example experiences tend to assume that they should build their specific chatbots using prompts – and ONLY prompts – engineered to elicit specific behaviours. While possible in many cases, there are at least three issues with this approach:

1. OpenAI interfaces are intended to be broad examples, and they could change, possibly upsetting the performance or operation of your own solution.
2. The costs of sustaining conversation awareness could be financially impractical unless you craft the solution with a mix of OpenAI APIs and other usual and customary technologies, such as embedding vectors, real-time data handlers, etc.
3. There are likely additional requirements that you haven’t yet encountered. Locking in your technical approach to a prompt-engineered strategy has a ceiling that is probably too low to address other challenges that are not explained in your post.

#3, BTW, is advice I give to my clients concerning the development of AI systems; always start with a requirements document even if it’s just a list of bullet points. This will also help us in the community to understand your objectives so that you get really good advice.

My Approach (in case it helps)

I created this example FAQ system using only a Google Spreadsheet and various Apps Script that create vectors for each Q/A. My web app vectorizes the new user question, finds the top three candidates in the vector store (i.e., the spreadsheet), and uses that content to create a completion prompt to generate the final answer for the customer.

While this is a simple Q&A process, it also maintains a conversational context by storing (in memory) a session object. This object contains the user’s questions, the answers we provide, and the relevant vectors. This allows me to monitor the conversation by generating an ongoing summary of the conversation as well as a list of keywords and entities that appear in the conversation as it unfolds.

For my FAQ requirements, the session object is not needed. However, it is needed for analytics and improving the FAQ content corpus (all customer interactions are tracked to improve the app). This approach could be used to provide a chat service with an ongoing memory of what has transpired.

I realize you’re not a coder, but for others reading this, tools such as LangChain and AutoGPT may also be helpful in building better chatbot systems.

Why Google Apps Script?

It’s not the most robust development platform, but I have found that it’s ideal for a lot of solution prototypes involving OpenAI and ultimately, my work is positioned to leverage Bard as well. I also see a massive swell of Google Workspaces customers wanting to accelerate their work with OpenAI. With that, I started a new substack to share these approaches and some code concerning my approaches.

[curt.kennedy]

Cool @bill.french , and extension for a CyberTruck!

Cool use of embeddings!

Only comment, is that I am easily able to attack your bot with a prompt injection:

[bill.french]

@curt.kennedy

I am easily able to attack your bot with a prompt injection:

LOL Funny. But to understand this, that injection does not impact any user except you, right?

[curt.kennedy]

@bill.french

LOL Funny. But to understand this, that injection does not impact any user except you, right?

That injection was just a nice LOL. But as long as your bot isn’t hooked to anything juicy on the back-end, I guess it can’t do much harm. Wait, I thought you were the AI security guy!

[bill.french]

@curt.kennedy

I thought you were the AI security guy!

Um, no. I’m not even “the security guy”. Most CISO’s don’t want to be seen in the same coffee shop as me. You may have me confused with someone who knows security.

@curt.kennedy

as long as your bot isn’t hooked to anything juicy on the back-end, I guess it can’t do much harm

It is hooked to an analytics gateway on the back end, but this is is an isolated system itself.

[bill.french]

This seeMicrosoft to behave pretty well -

[curt.kennedy]

OK, explain why it hallucinates so much …

These are the things I worry about public facing bots. Hallucinations (mostly) and secondarily prompt injections. Not saying it should be a big deal to you or everyone else. Just my concerns.

[bill.french]

@curt.kennedy

I worry about public facing bots

Indeed. This is a good test, mostly because it’s a prototype, but more concerning the speed that changes can be applied.

On the back side, there’s a real-time analytics process capturing every question and response. The responses are then vectorized to see how far from the core vectors there are in the FAQ corpus. Those deviations are then plotted and we see Cool McCoolface standing out against a backdrop of on-message questions. So spotting these is not difficult nor is updating the corpus (when new content needs to be added) and then quickly deploying the latest behaviors. But what is difficult is determining the definition of “off message”.

A small adjustment eliminated your hallucination…

But, the nature of definitive relevance to a subject is not so easily contained.