I'm having issues with gpt 3.5 turbo, as it refuses to answer the question, regardless of what I ask

Prompting
,
1

I have tried everything to make it work. From the simplest prompts and prompt chains to what I have now. I’m using gpt 3.5 turbo with temperature=0 and the other parameters set to default. I have tried other temperatures, but it also does not work. GPT-4 can easily answer the question.

Here is the full chat:

User: First, identify the language of the question. Do not answer the question yet. Just respond with the language (1 word):
Question: Qual a media de APIs por cliente?

Assistant: Portuguese

User: Now, translate the question to English. Do not answer the question yet. Answer in the following format:
Translation:

Assistant: Translation: What is the average number of APIs per client?

User: Now, write down all keywords of the translated question. Please start with KEYWORDS:

Assistant: KEYWORDS: average, number, APIs, client.

User: Write down a step by step to answer this question, if needed. Then, clearly state all information needed.

Assistant: Step-by-step to answer the question “What is the average number of APIs per client?”:

  1. Access the database containing information about APIs and clients.
  2. Retrieve the total number of APIs and the total number of clients.
  3. Calculate the average number of APIs per client by dividing the total number of APIs by the total number of clients.
  4. Return the result as the answer to the question.

Information needed:

  • Total number of APIs
  • Total number of clients

User: While analyzing the supplied knowledge base, endeavor to extrapolate or infer as much related information as possible to fulfill the following requirements. Even in cases where specific details are not directly accessible, aim to recognize potentially relevant information, making and stating logical assumptions when necessary. Do not deviate too much from the original quest.

The requirements:

  • Total number of APIs
  • Total number of clients

Knowledge base:
Wiipo, a fintech platform of Senior, offers three solutions that can help increase team productivity. The first solution, Wiipo Flex, combines all flexible benefits into a single card that is accepted in more than four million establishments. The second solution, Consignado Wiipo, is a simple and inclusive credit designed for companies with no risks or implementation costs. The third solution, Clube Wiipo, engages employees and partners with a club full of exclusive discounts and cashbacks of up to 50%. Wiipo’s solutions are designed for businesses of all sizes and can be accessed through the Senior X platform, which centralizes access to all management solutions and applications of a company. Senior X offers X Studio, which allows the creation and customization of solutions with no-code, low-code, and code technologies, and Senior X Store, an innovative collaborative ecosystem with access to more than 14,000 applications. Senior has a strong track record, with over 2.5 billion controlled accesses, 6 million payrolls processed per month, and 40 of the 100 largest construction companies in Brazil using Senior’s solutions.
Senior Sistemas is a company that provides technology solutions for enterprise management. They offer solutions for management, people, finance, and platform, including cloud services, software development, and consulting. One of their highlighted solutions is Senior XPlatform, which simplifies and streamlines management processes. In addition, they offer several products and services, such as Consignado Wiipo, which is a simple, inclusive credit designed for companies with no risks or implementation costs, and Clube Wiipo, which engages employees and partners with a club full of exclusive discounts and cashbacks of up to 50%. Senior X is a platform that centralizes access to all management solutions and applications of a company, with immediate integration between different business systems and processes. It also offers X Studio, which allows the creation and customization of solutions with no-code, low-code, and code technologies, and Senior X Store, an innovative collaborative ecosystem with access to more than 14,000 personalized APIs. The Senior Sistemas blog provides updated content written by experts on business trends and learnings. The website also answers frequently asked questions about enterprise management systems, such as what they are, the types of systems available, and what ERP is. Finally, the website highlights that over 11,000 clients, including some of the largest companies in various industries, have transformed their management with Senior’s solutions.
Senior Sistemas, a company that provides solutions for management, people, finance, and platform, has a website that offers a range of features. The website includes a whistleblowing channel, news section, blog, library, and press room. Customers can also access a client portal, support, and forums for product discussions. The company offers various products, including ERP, HCM, Wiipo, WMS, CRM, and Ronda Senior. Investors can find a dedicated section on the website, and job seekers can visit the career page to find job postings and submit their resumes. The website is copyrighted by Senior Sistemas and developed by Brivia. Additionally, the website includes information about Senior’s highlighted solution called Senior XPlatform that simplifies and streamlines management processes. Wiipo, a fintech platform that offers flexible financial services and benefits, is also featured on the website. The website also provides contact information and a blog with articles related to business management.

Assistant: Total number of APIs and total number of clients are not directly available in the knowledge base. It is possible to make some assumptions based on the information provided, but they may not be accurate. Here are some possible assumptions:

  • Wiipo offers three solutions: Wiipo Flex, Consignado Wiipo, and Clube Wiipo. It is possible that each solution has its own set of APIs, so the total number of APIs would depend on how many APIs each solution has. However, the knowledge base does not provide this information.
  • Senior XPlatform, which is highlighted as a solution that simplifies and streamlines management processes, offers X Studio, which allows the creation and customization of solutions with no-code, low-code, and code technologies, and Senior X Store, an innovative collaborative ecosystem with access to more than 14,000 personalized APIs. It is possible that the 14,000 personalized APIs are available to all clients who use Senior XPlatform, but it is not clear how many clients use this platform.
  • Senior Sistemas has over 11,000 clients, including some of the largest companies in various industries, who have transformed their management with Senior’s solutions. It is possible that the total number of clients who use Senior’s solutions is higher than 11,000, but this information is not available in the knowledge base.

In summary, based on the information provided in the knowledge base, it is not possible to determine the total number of APIs or the total number of clients.

User: Given these assumptions and the step by step built earlier, apply it and answer the original question. You have to answer the question. Do not refuse to answer the question. Does not matter whether it is accurate or not. You have no choice but to answer it. Start your response with ANSWER:

Assistant: ANSWER: Based on the information available in the knowledge base, it is not possible to determine the average number of APIs per client. The total number of APIs and the total number of clients are not directly available, and the assumptions made earlier may not be accurate.

1 Like
2

Change the temperature for sure move to 0.2-0.3 so it can get a bit creative – I think it is not picking up personalized APIs as APIs.

3

@jethro.adeniran I tried around 10 different values for temperature, from 0 to 1.25 already

4

What do you mean when you say it didn’t answer the question?

It looks to be that it did answer the question,

5

@elmstedt It indeed answered the question. My issue is that regardless of instructions asking to ignore inaccuracies and try its best to estimate the answer, including assumptions required, it was not capable of obeying. The same did not happen with GPT4, as it would understand the prompt easily, make the assumptions required and return a proper answer. GPT3.5-turbo doesn’t even try.

6

I feel like even the API models have been in degradation. In the past, this type of task would be almost trivial for GPT3.5-turbo

1 Like
7

GPT3.5 will usually not usually answer ambiguous questions or questions with potentially risky outcomes. Asking it to make assumptions for missing data and then build a solution from the assumptions of data that does not exist is deeply nested task, that seems a little to far for GPT-3.5 to infer.

8

I have experienced the exact same problem with 3.5.
You can 100% fix this by providing 1 or 2 worked examples in your prompt.

2 Likes
9

What kind of examples would you give to allow these relatively straightforward general questions to be solved?

10

Create similar examples to your specific use case.

You can use GPT-4 to generate synthetic examples (then use for GPT-3.5 prompt) to get the ball rolling.

1 Like
11

don’t use any app from playstore symply go to openai.com creat an account and bookmark the website that’s only how its woked.

12

you need to teach it how you are getting the total number of APIS and CLIENTS from the scenario you gave it. Then ask again and you’ll get the correct answer

13

how did you* get the # of apis and clients from reading the knowledge base? Teach it your system you used.