Gpt-4 is getting hallicunated

API
1

Hello,

I am using gpt-4 API. I have uploaded the excel sheet and asked the question from the excel. Well Out of 10 questions I am getting perfect response for 9 questions but I am struggling to get the proper response for the 1 question.

Well I have asked the questions such as highest sales, lowest sales , sales percentage between the two periods etc, and i am getting the perfect response. But when I am asking what is the highest value in particular period . then it is hallicunating. It’s weird though.

    Model = "gpt-4",
    Temperature = .7,
    MaxTokens = 500,
    Messages = new List<ChatMessage>

new ChatMessage
{
    Role = ChatMessageRole.System,
    Content = "You are a helpful assistant."
},
new ChatMessage
{
    Role = ChatMessageRole.User,
    Content = $"Based on the following text from the Excel file, answer the question.Excel Text:{extractedExcelText} Question: {question}"
}

Any help or suggestion on this???

2

If you want the model to crunch numbers, it’s probably best to give it a calculator.

Give it a tool that allows it to do what you want it to do. If it’s simple stuff, you might be able to get away with a Chain of Though approach :slight_smile:

2 Likes
3

well… Can you give the example for Chain of Thought of how to change here ? Sorry , its just been a week when I have started working on it. I am new here

4

What’s your prompt so far?

ah.

well, you could ask it to first write out the numbers it needs to compare, and then come to a conclusion based on that list, for example.

5

It didn’t work properly :frowning:

6

I’ve found it is always better to chop up as much of your data as possible before sending it to be processed, as opposed to asking it to perform multiple ‘operations’ at once. For example, you would be much better off only passing the data for the period you are interested in and asking it to find the highest value, instead of passing it the data and asking ‘what is the highest value in X period’. It takes more time upfront, but I have found that over the long run it saves a lot of headaches.

1 Like
7

well ,… it is checking the data of that period only but it is giving some other value . For example
image

Which items has the highest value in the duration 6?
instead of 2470 it is showing as 1875

8

Again, the answer has already been given. If you want the model to do math, do it in the python tool.

1 Like
9

So tell me one thing. When I am asking which duration has the highest or lowest profit in the sheet then it is giving the proper response. It is also comparing and giving the sales percentage of this year comparing to the last year .I haven’t perform any math in the backend code. Then what is the use of math ? My data is not complex

10

Asking for a minimum of maximum is math.

Your question has been answered—use the Python tool.

1 Like
11

Since GPT is a text generation model and not a calculation tool, it’s better not to make GPT do the calculations.
If you’re interested, you might find it interesting to look at my previous threads.

12

So how should I calculate . Do I need to create the calculator ? if I have to create the calculator and call in the GPT functionality , then yes I have created the calculator. Well , I am getting the response now but again it is hallucinating sometimes. Let’s say out of 5 questions of min and max , it is hallucinating at least for once. And I am using c#

13

So how should I calculate . Do I need to create the calculator ? if I have to create the calculator and call in the GPT functionality , then yes I have created the calculator. Well , I am getting the response now but again it is hallucinating sometimes. Let’s say out of 5 questions of min and max , it is hallucinating at least for once. And I am using c#

14

As many suggested, GPT 4 is not ideal for math. Use Python or whatever language you are familiar with. Preprocess the data and do the math with a programming language and then pass the preprocessed data to GPT 4 for interpretation for example.

In your particular case, we have found that the model is “lazy” by default. So, for example, it might stop looking at rows at a certain point and give you the highest value of the few rows it looked at. To confirm if that is the case, rewrite your prompt as The following array has X values (for example, 9); look at every number before providing your answer. Which one is the highest number? - We previously used this approach with preprocessed JSON files with 15,000 rows and noticed the model was lazy. Providing the additional info such as each row has an index number from 1 to 15,000, look at every row, etc. It works better.

2 Likes
15

Thanks for the clear classification. I performed the same way

16

Using AI on Excel sheets is the most overrated thing ever.

Use ChatGPt to learn SQL goddamn!

2 Likes
17

I did that with a Google Sheet module and version 3.5 but I don’t understand since the answer to my questions in the chat was totally different from that obtained in Google Sheet

and otherwise I wanted to ask you if I use the new version in Google sheet can it process images for me?