Remembering previous information in the tech support bot

API
,
1

Hello, everyone!
I’m making a chatbot that could answer customer questions based on information from company documentation (not always technical, product information is included).

I do everything as in the example from the example in the official githab at:
openai-cookbook/examples
/Question_answering_using_embeddings.ipynb
(I can’t post links)

That is, I downloaded articles with instructions and information about products from the company’s website (in my example, a cell phone company), added embeddings and searched for the closest article for each query.

Here’s an example of the work:

How much is the All-in-One rate?

All-in-One is $10 a month for the first three months and then $13 a month thereafter.

What’s included?

Unfortunately, I don’t know which plan we’re talking about. Please be more specific about which plan you are asking about so I can give you a more accurate answer.

As you can see, GPT has not remembered previous posts. I would like to point out that everything else works perfectly. It answers questions well based on well chosen articles.

I make such a request with a python command:

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
        temperature=0
    )

Where the messages are:

{'role': 'system', 'content': "You're answering questions from customers of an 'Eggplant' service provider."}
{'role': 'user', 'content': 'How much is the All-in-One rate?'}
{'role': 'assistant', 'content': 'All-in-One is $10 a month for the first three months and then $13 a month thereafter.'}
{'role': 'user', 'content': '<The line is too long. I posted it below>'}

Content:

Use the Eggplant operator knowledge base and message history below to answer the following customer question. Clarify the details, if necessary. Do not refer to the Eggplant articles. If the answer cannot be found in the articles, write "I could not find the answer".

Eggplant knowledge base article:
"""
How to change my tariff?

Services

Mobile communications

Under Tariff or Subscription.
"""
...
<Lots of articles>
...

Article from the Eggplant knowledge base:
"""
Included in the subscription

Details about All-in-One tariff

Mobile communications

Tariffs

Calls and SMS
Home internet and digital TV
On-line cinema
Unlimited social networks and messengers
Unlimited Internet access while watching On-line cinema
Subscription is valid in Eggplant network on the territory of Russia.
For the scope of services provided as part of the subscription, see the Detailed Terms and Conditions.
"""

....

...
<Lots of articles>
...

Customer question: What does it include?

How can this be fixed?

Or, can you please tell me the best way to do it? I thought about fine-tuning, but as far as I understand, fine-tuning is for text completion, not answering questions.

Thanks!

2

Welcome to the forum!

I think this comes down to the use of the word “rate” and not “plan” in telephony a rate is not the same as a plan, so when you asked what the all in one rate is the system looked up the cost, but does not seem to be treating that as a plan, and so, requests details of the plan. If you expect customers to use rate and plan interchangeably, you must specify that in the prompt.

1 Like
3

Thanks for the reply!

I’m actually using a different language where the word is a clear word definition of payment. I used a translator to query here. So that’s probably not the problem. Do you think this way of implementation is generally okay? Or are there better options?
Fine-tuning requires a history of correspondence between customers and operators. But I do not have this history, so i can only rely on the knowledge base.

4

I think you might find this course and perhaps the associated one very useful, Andy Ng and OpenAI’s own Isa Fulford go in to detail about creating business orientated chat bots and some of the methods for achieving high quality results.

2 Likes
5

Thank you so much! I’ll be looking into it.

1 Like