Math is a known weak point of LLMs. A simple rule is not using the LLM (ChatGPT) for something that can be achieved perfectly with a standard calculator/computer.
That said, I think this “math paradox,” where our seemingly most advanced computer cannot perform arithmetic, does a great job demonstrating the paradigm shift brought to us by AI.
In short, our past computers were all like “math” people. But now, “creatives” have joined the party.
Below is an excerpt from a blog post I wrote yesterday that tries to address this:
while ChatGPT may appear capable of solving simple arithmetic and some algebraic equations, it regularly fails with more complex math problems.
This inability to solve math problems may seem paradoxical because our calculators and computers have been successfully helping us solve math problems for over 50 years. So why can’t AI, running on a computer, solve the same equations?
This paradox is an excellent opportunity to recognize the fundamental shift in how the technology underlying our interactions with computers has changed. For example, it’s common for humans to refer to others based on their abilities, like being a “math person” or a “creative.” Each designation comes loaded with expectations we have of those types of people.
Until now, computers have always been the “math person” who could be relied on for calculations, but not the “creative” you would go to for works of art. Technology like ChatGPT is our first glimpse at what it might look like if computers played the role of “creative.”
To understand ChatGPT’s math capabilities, imagine you need a math tutor. But for the sake of this discussion, you decide to go to the “creative” person for help. Assuming they know the math required, the “creative” will have particular strengths in helping you, like communicating the algorithm in easy-to-understand language instead of mathematical notation. But coming up with an exact answer is still something they rely on a calculator to complete.
Suppose you need to use ChatGPT for math assistance. In that case, ask for instructions on solving a math question rather than relying solely on its final answer. For example, I have witnessed ChatGPT providing the correct formula while giving a wrong answer after the equals sign. So, it’s best to use ChatGPT for math assistance with some caution. And always use a traditional calculator to check your final answer!