I asked ChatGPT to calculate the speed of light in furlongs per fortnight and it was off by 18 orders of magnitude. See screen capture.
The correct value is 1.8 E12 furlongs per fortnight.
Mike
The correct value is 1.8 E12 furlongs per fortnight.
Mike
1 Like
- Don’t use ChatGPT 3.5 for math
- Don’t depend on LLMs to solve math by themself
- Use ChatGPT4 with
Code Interpreterfor math
4 Likes
Welcome to the Dev Community, great to see you playing around with ChatGPT and testing it’s capabilities!
As @RonaldGRuckus and @vb both eluded to, it’s important to understand the limitations of the models, which it seems like you’re already finding out.
GPT-4 is much stronger at logic and reasoning tasks, plus it has “code interpreter” (aka Advanced Data Analysis) which it a tool it can use to write and run code to solve problems.
These models are made to predict the next token (a sub-section of a word used in the transformers architecture they’re built on). Predicting the next token can get you some phenomenal results in natural language processing, but it makes more complex concepts like math pretty hard.
GPT-4 is smart enough to get around this issue through the use of code-interpreter. This way it can just predict the next token to write the code, then run the code and provide you with the result!
If you’re interested in learning more about how they work there are tons of resources online, as well as old topics on here that I encourage you to read through.
Exciting times!
3 Likes
Proof that LLMs have not killed off traditional compute (and may not for a long time), but are simply an additional tool in our ever growing set …
4 Likes
In ChatGPT’s defense, questions like “calculate the speed of light using in furlongs per fortnight” are exactly the kinds of Math questions that make me feel terrible at Math.
I would look at that problem, stand up from my desk, and walk out immediately before I become “that guy” that asks “I thought fortnight was a video game?”
2 Likes