Anyone else using o1 constantly for actual math (especially pure math)? What is your experience so far?
5 Likes
Although I’m not using it for pure or advanced math, I’ve been extensively using AI for school-level math, especially to generate and solve questions or create exams for basic education students.
Since GPT-3.5, I’ve also tried other AIs like Gemini (since Bard), Copilot (since Bing Chat), Claude, Meta, etc., and they all struggled significantly. They often got lost in calculations, generated unsolvable questions, or failed to solve existing ones correctly.
However, the current O1 (final) seems to be the most accurate so far. In my use cases, it has successfully generated high-quality questions, modified them as needed, and solved existing ones without errors up to this point.
PS: I’m using ChatGPT Plus (not Pro)
2 Likes
Wow! Fascinating! Please post alink if you ever decide to document your journey
That has been my experience as well, except I’m working with university-level math. One thing that is very common in the other models (Gemini, Claude, etc.) is that they struggle to simplify fractions, but they also make a lot of other different mistakes as well, which makes sense since they can’t reason like o1 can.
I’m very excited for the future. Once o1 starts to support all kind of file formats (not only images), it will be even better. I also expect that the reasoning will improve over time.
I play with o1 literally all day. It’s so fascinating to see it reason and write precise proofs. It’s such wonderful technology. 2-3 years ago I didn’t think something like this could exist at all.
Nice! The Mandelbrot set is beautiful.
I would say 4o is very good for the really basic problems. The more advanced the math is, the more it struggles, though. o1-preview was a bit of pain because it didn’t support images, but now with the full o1 supporting images, it’s so much easier.
1 Like
Your mandelbrot sets! Not that I’m not interested in andremr’s school journey. Maybe there could be a group called GPT for pure math.
The only area of school mathematics where I haven’t found good AI options is geometry. It’s not possible to generate geometry questions that involve images, only theory. Also, the model that reads images, 4o, has trouble interpreting various types of images, such as those involving angles, geometric solids, or the division of shapes into other shapes. When dealing exclusively with theoretical statements or problem descriptions, model O1 works well, but many students, especially younger ones, need to work with images.
If I’m not mistaken, there still isn’t an AI capable of generating precise images, like geometric shapes or technical drawings. Is there anything like that?
I have the same dream - for AI to be able to generate precise math images. I found a workaround that works to some extent: ask GPT-4o to use code interpreter to generate the image you want (advanced analysis or something like that is the new name of it).
While o1 is better at coding but can’t run script. A workaround is to ask it to generate the python script to draw the image and then run it outside of ChatGPT on your computer directly. Not sure how good is this approach, I use it when I want to understand a math concept and I ask it to illustrate it, but nothing like complex geometry. But if anything has a chance of working for that then it’s via code generation.
If you try it, please share your experience with it: to what extent does it work?
I’ve been excited to use it to explore a recreational math problem but have found it gets lost much faster than I expected. The recreational mathematics I was exploring involve two overlapping disks of a certain radius which have the ability to rotate to fixed positions, ie N=3 positions per disk means they rotate in 120 degree increments. There are some terms we use in the world of sequential movement puzzles that I have to define for him, and then I ask some simple questions, such as, “Given two disks with centers 2 units apart and a radius of 1.1 each, a small overlap is formed. If we made a physical copy of this puzzle with the disks allowed to move in 120 degree increments, how many total pieces would the puzzle need to allow all arbitrary sequences of moves?”
It turns out that in this simple case there should be 7 pieces. But for different N and different radii R, things get a lot more interesting, fast. In fact in some cases the number of pieces goes infinite suddenly at some value of R. And in those cases there is unexpectedly beautiful geometry that arises, including some legitimately self similar regions with infinite zoom as well as some penrose tiling patterns on the N =5 disks. I’m hoping for better reasoning in o3.
Just one example failure was having the model imagine three consecutive moves on one disk, which rotates the overlap piece right back where it started. The model couldn’t conceive this and reasoned that this must be some new position that was physically similar but distinct from the starting position.
Some of this is on me I’m sure. More practice needed in formulating things clearly. If anyone would like to see the chat I will share it. And if allowed, some discussion of the problem on my old blog https://www.deluxerender.com/2015/12/gizmo-fractals-in-2d/