How can I make GPT understand the user role better? Here is an example to illustrate the problem. The following prompt to Chatgpt (3.5) comes back as “In the customer support ticket, Markus is the customer, and Matthew is the support agent.”. This is our typical scenario where the email signature of agents will identify them as support agent but customer’s signature won’t usually have anything specific to identify them as a customer.
In the following customer support ticket, who is the agent and who is the customer?
Ticket Details:
Title: Issue with my printer
Description: My ZenX Printer 2000 is not turning on
Email:
From: Markus
To: Matthew
Subject: What’s your model number and serial number
Body:
Hi Matthew,
Hope you are doing well. Can I have the model and serial number so I can verify the warranty? Regards,
Markus
Support Agent.
Email:
From: Matthew
To: Markus
Subject: RE: What’s your model number and serial number
Body:
Hi Markus, I’m well. It’s A01234 and 1144d. I tried restarting as well but no luck.
Regards,
Matthew
I just dumped this into ChatGPT and it did well - up to the point of answering wrong, maybe it would be better with a custom instruction telling the AI that it has a special purpose: “You are an AI assistant, designed for identifying the roles in unlabeled emails. Pay close attention to the contents.”
Prompt
Email exchange:
Ticket Details:
Title: Issue with my printer
Description: My ZenX Printer 2000 is not turning on
Email:
From: Markus
To: Matthew
Subject: What’s your model number and serial number
Body:
Hi Matthew,
Hope you are doing well. Can I have the model and serial number so I can verify the warranty? Regards,
Markus
Support Agent.
Email:
From: Matthew
To: Markus
Subject: RE: What’s your model number and serial number
Body:
Hi Markus, I’m well. It’s A01234 and 1144d. I tried restarting as well but no luck.
Regards,
Matthew
Task 1: From the email exchange above, we need to separate out what each person said. Produce a list of ALL messages sent from one party, prefixing their name before each email they sent. Then switch to emails from the other writer and produce a list of ALL messages that the other person sent.
Task 1 output format (without email subjects)
<sender_1_name>
<message_1>
<message_2>
…
<sender_2_name>
<message_1>
<message_2>
…
Task 2: From reading the clear sender presentation you’ve produced, you can now answer the ultimate question: Who is the customer, and who is the company support representative?
Output, ChatGPT 3.5
Task 1:
Markus
Hi Matthew,
Hope you are doing well. Can I have the model and serial number so I can verify the warranty? Regards,
Markus
Support Agent.
Matthew
Hi Markus, I’m well. It’s A01234 and 1144d. I tried restarting as well but no luck.
Regards,
Matthew
Task 2:
Markus is the customer, and Matthew is the company support representative.
AND for people that say “show me the evidence that the AI is dumber”. We go back to gpt-3.5-turbo-0301 by API – Correct answer:
Task 1 output:
Markus
Hi Matthew,
Hope you are doing well. Can I have the model and serial number so I can verify the warranty? Regards,
Markus
Support Agent.
Matthew
Hi Markus, I’m well. It’s A01234 and 1144d. I tried restarting as well but no luck.
Regards,
Matthew
Task 2 output:
Markus is the company support representative and Matthew is the customer.
Output, GPT-4 API
Task 1:
Markus
Hi Matthew, Hope you are doing well. Can I have the model and serial number so I can verify the warranty? Regards, Markus Support Agent.
Matthew
Hi Markus, I’m well. It’s A01234 and 1144d. I tried restarting as well but no luck. Regards, Matthew
Task 2:
The customer is Matthew and the company support representative is Markus.
It takes an AI with an intelligence upgrade to answer this input correctly.