How do I identify if a Question is being asked from a few paragraphs of text

Prompting
1

Hi OpenAI community,

I have spent countless days trying to craft a prompt that reliably identifies any Questions being asked out a few paragraphs of text. I have had limited success but as soon as I test it on more examples I see that the accuracy isn’t actually all that good.

Example prompt:

As an intelligent text analysis bot this is an email sent to Person A to Person B to analyze:
“Paragraphs of text …”
Who is asking for what?
1.

I have tried many other variants like:

Is person A asking person B any interrogative questions? Say Yes or No. If Yes then extract only the interrogative questions that require an answer and list them in point form.
Is person A asking person B a question? If person A is asking person B a question then extract the exact text of the question.
Who is asking for what and from whom?1.

Sometimes Divinci (with a low temperature) works well but other times it OpenAI takes the text which may be an answer to a previous question and tries to work out what question was originally being asked.

Ideally, I would actually like to directly quote the text it identifies as Questions but I have not been able to get OpenAI to extract and quote text.

If you can point me in the right direction, I would appreciate it.

Thanks in advance!

2

Even as a human it is difficult to parse this prompt. The cardinal rule for prompt engineering is “keep it simple”

image
image1667×334 27.8 KB

List any questions asked in the following passage

PASSAGE:

Are men less attracted to women with tattoos? I hear all the time that tattoos are a turn off for a lot of men. However, I am pretty covered as far as tattoos go and i get far more male attention now than i did when i didn’t have any tattoos. Is it true tattoos make a women less attractive or is it the opposite?

QUESTIONS:
-Are men less attracted to women with tattoos?
-Do tattoos make a women less attractive?

It doesn’t seem to work if there aren’t questions, though it can infer what the question asked might have been:

image
image1745×360 42 KB

Thought it does work if you remove the leading hyphen

image
image1702×432 27.7 KB

3

Try it as a one shot.

Here is the prompt I would try:

Extract all questions from the falling paragraph:

  1. question 1
  2. question 2

Extract all questions from the falling paragraph:

1.

4

Do you really mean to detect questions, or rather “requests”? There are many ways to ask a question and some don’t even have a question mark. Examples:

  1. Can we have the document by Friday? // obvious question
  2. Can we have the document by Friday. // a question but sender forgot the question mark
  3. Fred, please send me the document by Friday. // a request - thus a pseudo question
  4. Would be great to get the document by Friday. // a request - thus a pseudo question
  5. How are you? // looks like question but isn’t a real one
  6. We are all guilty, aren’t we? // not a question, yet it has a question mark

This prompt works very well with davinci but not so good with lower models (bold is the completion)

List any requests in the following passage:

PASSAGE: How are you? We’ve been seeing good progress in the last two weeks. By when can we have the floor plan ready? Can we have the flow chart by Friday? Fred, please send me the contract by Saturday. Would be great to get the hammer by Sunday. We are all guilty, aren’t we?

REQUESTS:

- Can we have the floor plan ready by when?
- Can we have the flow chart by Friday?
- Please send me the contract by Saturday.
- Would be great to get the hammer by Sunday.

5

I have been trying something similar, I have found that DaVinci with a low temperature seems to be working the best.

I would also take a look at your formatting. I am wondering if you are sending the email header information? This might help the bot determine who is talking to who.

I tried this prompt on text-davinci-002, with 256 max length, ## as stop sequence, and 0 temperature.

Read the emails and answer the questions based on the emails. If you cannot answer the question, answer “No answer”.

Title: How does this time work?
To: John Smith john.smith@gmail.com, Becky Joe becky.joe@gmail.com
From: Jane Doe jane.doe@gmail.com
Body: What times work best for you both meet? And John do you have the report yet?

Title: Re:How does this time work?
To: Jane Doe jane.doe@gmail.com, Becky Joe becky.joe@gmail.com
From: John Smith john.smith@gmail.com
Body: Does 10 am Tuesday work for you both? No, I don’t have the reports yet

Questions:

  1. List the questions found in each email provide who asked the question, who is assigned to answer the question, and the email title.

Answers:
1.

Sample response:

  1. What times work best for you both meet? - Jane Doe, John Smith, How does this time work?
  2. John do you have the report yet? - Jane Doe, John Smith, How does this time work?
  3. Does 10 am Tuesday work for you both? - John Smith, Jane Doe, Re:How does this time work?
  4. No, I don’t have the reports yet - John Smith, Jane Doe, Re:How does this time work?

I now that formatting is important, maybe rather than a list you ask for a table?

You can ask it to create a table based on the answers.

Example:

Create a table of the answers above where the columns are question, email title, who asked the question, and who is assigned to answer the question.

Table:

Example Answer:

Question | Email Title | Who Asked the Question | Who is Assigned to Answer the Question

  1. What times work best for you both meet? | How does this time work? | Jane Doe | John Smith and Becky Joe
  2. Does 10 am Tuesday work for you both? | Re:How does this time work? | John Smith | Jane Doe and Becky Joe
  3. And John do you have the report yet? | How does this time work? | Jane Doe | John Smith
  4. Becky, what did the client say about the project? | How does this time work? | Jane Doe | Becky Joe

A table would let you create a csv, or array, and do all sorts of things programmatically.