I’ve been working on using GPT-4(Through ChatGPT Plus) to rephrase and change all the numerical values and names of mathematical questions, and to provide me with step by step solution for the rephrased questions so that I can easily replicate mathematical questions of the same type for providing adequate practice to students within no time and minimal effort.
During this… I encountered something weird. I had a question for which I wanted GPT-4 to provide me step by step solution. The question is “What are all the prime factors of the number 2457?”
No matter how many times I open up a new chat and ask the same question GPT-4 manages to get it wrong every time. GPT-3.5 can do this question with no problems.
Here’s one of the responses that I got…
I have noticed the following patterns. It says every time that 2457 isn’t divisible by 3. That has remained constant in every answer. In one of the responses, it even calculated… adding 2 + 4 + 5 + 7, and saying that the result is 18, and 18 is divisible by 3 so 2457 IS NOT DIVISIBLE BY 3. It contradicted itself in the same sentence but wouldn’t admit that 2457 is divisible by 3. In other questions mainly, it divides the number 1457 by 3 and says that it is equal to 819.0, which is not a whole number and is therefore not divisible by 3(As shown in the image above). It even told me once that 2457 is a prime number. I haven’t tested this with other numbers yet, but it is a really weird and interesting problem.