Visualizing simple graphs and exact structures poses an issue for the gpt

ChatGPT
,
1

I am trying to use chatgpt for mathematics-physics related problemsolving, and it fails remarkably with visualization.

We had a nice conversation and understanding with chatgpt about representations of 3-dimensional triangulations in the dual as nodes (representing tetrahedra) and links (representing faces). It understands well the structure, and we can discuss changes to the graph.

The task then i gave it was to make a picture of a single node with 4 links coming out of it (its a tetrahedron in the dual) and it failed remarkably, it produced many unnecessary stuff on the picture, and was not able to make this exact simple thing: a dot with 4 links coming out of it.

I wonder if chatgpt can be used in the future for graph-production, or it can do only artistic images?

1 Like
2

Welcome to the forum!

In using ChatGPT with math related problems for the past two years here are some of my conclusions:

  • LLMs (non-multimodal) work best with text representations. In other words if you are drawing graphs with nodes instead use a representation of the graph as text, something like GraphViz might work. LMMs will understand this much better than an image. Personally I might try an image question with a new model to see how it does but at present results are not consistent and accurate enough to be practical.

  • LLMs (any mainstream) do not reason logically. Yes the o1 is a step forward in that direction but even then not consistent enough to use daily for such.

  • LLMs are much better with creating programs for structured images. Instead of asking ChatGPT to create an image relying on DALL*E, ask for code that can create the image.

  • Consider OpenAI evals.
    From AMA: More generally, what are we as developers not doing as much as you think we should?
    Reply by @michellep

    One big one is evals! I see tons of developers not using evals at all and relying on vibes for rolling out changes to prod. Would highly recommend creating some simple evals using our evals product (or open source offerings) so you can update with confidence when we release new models.

    Yes I know evals don’t work for image generation at this time but this combined with the other ideas could unlock a practical path forward.