# HOW to extract data from a graph image with ChatGpt4

**URL:** <https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881>\
**Category:** Prompting\
**Tags:** gpt-4\
**Created:** [January 31, 2024, 1:48pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881 "2024-01-31T13:48:21Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![guilhem.dezanneau](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@guilhem.dezanneau](https://community.openai.com/u/guilhem.dezanneau)\
**Post date:** [January 31, 2024, 1:48pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/1 "2024-01-31T13:48:21Z")

</div>

Hello,

I am new to chatgpt but I found it very nice to extract some information from images. Then I decided that it should be possible to extract data from a graph. From what I saw, everything is here to make it possible but the system always fails at a given moment (never the same moment) and never finishes the action.

Here is the prompt I submitted to extract the information.  
“My objective is to extract the data from a graph in form of an image.Here is an image representing a graph with two curves. one curve is made of blue triangles markers connected with a blue dash line, the second one is a curve made of red circles markers connected with a red dash line. These curves represent the log(sigma_T) where sigma is the conductivity and T the temperature in kelvin, as a function of 1000/ T. First, can you identify the two compounds that correspond to these two curves. Their composition is indicated in the title of the image and in the legend with different x values for barium content. Then make me validate before going on. Second, can you identify the blue triangles markers and red circles markers in the image. Each triangle or circle should contain several hundreds of pixels. So use this information to remove small clusters. Remove also the markers that contain empty regions or that have a shape of a letter or a number. Then show me on the image the centers of those clusters consisting of triangle and circles. make me validate before going on. Finally, extract the values associated with the blue triangles markers and the red circles markers and put them in a table where the first column is the x position in th 1000/T scale, where the y position is the log(sigma_T) value. Then make a graph with the extracted value similar to the original one. put the graph close to each other to allow for a comparison.”

It detects quite correctly that there are two curves, one made of blue triangles and one made of red circles and what they correspond to, but it never succeed in performing the process the good way, it fails very often. I have spent 15 days on this without getting my final objective that is simply to recover the data of the curves. The point also is that the extraction procedure seems to vary from an attempt to another, and chatgpt misses important information in the prompt.

 ![one_graph_1](https://us1.discourse-cdn.com/openai1/original/4X/7/3/f/73fa8ad27252fd4cc6e690549c25eeffee2c0a03.jpeg)

---

<div class="post-metadata">

**Author:** ![PaulBellow](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/paulbellow/32/597962_2.png) [@PaulBellow](https://community.openai.com/u/PaulBellow)\
**Post date:** [February 1, 2024, 7:35am UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/2 "2024-02-01T07:35:06Z")

</div>

I don’t think the tech is there yet to be able to achieve what you want. Even a game grid was difficult…

See:

> [@Issues with Recognizing Units on Hexagonal Grid](https://community.openai.com/t/issues-with-recognizing-units-on-hexagonal-grid/434479):
>
> Hey guys, I am using the new image recognition feature and it generally works very well. I was surprised that a task I thought was easy is instead failing continuously. This is the prompt and the image. Looking for insights. Any ideas? “You are an expert analyst of hexagonal game maps, keeping in mind that the hexagonal grid starts from B1 and has a maximum of 4 rows (so max B4) and has 17 columns (so it goes up to R) and they go from top to bottom (i.e., B1 is at the top and descends towards …

That said, the tech will continue improving. You might wait for a newer model (Guessing you’re using GPT-4?)…

It’ll be possible eventually.

What’s your prompt look like for it?

---

<div class="post-metadata">

**Author:** ![guilhem.dezanneau](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@guilhem.dezanneau](https://community.openai.com/u/guilhem.dezanneau)\
**Post date:** [February 1, 2024, 8:48pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/3 "2024-02-01T20:48:38Z")

</div>

I put the prompt in the original message.  
Maybe you did not see because it does not look like a typical prompt. Sorry for this, I am quite new in this field.

---

<div class="post-metadata">

**Author:** ![cdonvd0s](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/cdonvd0s/32/294704_2.png) [@cdonvd0s](https://community.openai.com/u/cdonvd0s)\
**Post date:** [February 1, 2024, 8:52pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/4 "2024-02-01T20:52:55Z")

</div>

It’s better to start with “What do you see in this image, respond with a detailed answer” (Not the best prompt)

If you don’t see something close to what you are looking for after a few tries, it’s safe to assume that it cannot do it right now.

The Mantra is “Just wait” till the next model drops.

---

<div class="post-metadata">

**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\
**Post date:** [February 1, 2024, 8:54pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/5 "2024-02-01T20:54:50Z")

</div>

Why not use the LLM to extract the **semantics** of the graph, and then use an OCR to extract the graph markings and values via bounding boxes?

This could probably be done without GPT-4v, honestly.

---

<div class="post-metadata">

**Author:** ![N2U](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/n2u/32/54438_2.png) [@N2U](https://community.openai.com/u/N2U)\
**Post date:** [February 1, 2024, 9:02pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/6 "2024-02-01T21:02:04Z")

</div>

Yeah, but the accuracy simply isn’t there yet 😕

Even if you’re able to extract the values somewhat adequately, they will still be far less accurate than the original measurements.

I highly recommend reaching out to the original author of the research and simply asking them for data. They are usually more than happy to help. 😆

---

<div class="post-metadata">

**Author:** ![guilhem.dezanneau](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@guilhem.dezanneau](https://community.openai.com/u/guilhem.dezanneau)\
**Post date:** [February 2, 2024, 10:26am UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/7 "2024-02-02T10:26:27Z")

</div>

Hello. My objective is obviously not to have those data in particular but to explore a strategy to extract them from graphs. As researcher, I knwo that I can ask to the authors, but I do not want to ask for tens of authors if I want to extract data from a significant amount of articles.

Thanks anyway for your suggestion.  
Regards  
Guilhem

---

<div class="post-metadata">

**Author:** ![guilhem.dezanneau](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@guilhem.dezanneau](https://community.openai.com/u/guilhem.dezanneau)\
**Post date:** [February 2, 2024, 10:28am UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/8 "2024-02-02T10:28:15Z")

</div>

LLM and OCR is ok to extract the text information in the graph like material identification, activation energy for slopes, doping levels etc…

Failure occurs when I ask for markers detection (usually I have to ask for additional things to remove legend or other clusters detected) and even more for position calibration.

---

<div class="post-metadata">

**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\
**Post date:** [February 2, 2024, 4:57pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/9 "2024-02-02T16:57:46Z")

</div>

> [@guilhem.dezanneau](#):
>
> Failure occurs when I ask for markers detection (usually I have to ask for additional things to remove legend or other clusters detected) and even more for position calibration.

The issue with your approach (in my opinion) is that you are using a tool that doesn’t offer any ability to tune besides some prompt engineering, and it also doesn’t return the coordinates of anything like a typical OCR does.

You can use an OCR to extract the values and can even try fine-tuning it to capture the symbols. You can correlate the bounding boxes to find the values.

 ![Screenshot from 2024-02-02 11-55-45](https://us1.discourse-cdn.com/openai1/original/4X/5/1/9/5199a69c8785d4cc4535b985171f5ed798c2b324.png)

Then you can use simple logic to capture the symbols and correlate them on the axis for the value. You can also use an LLM to understand the semantics of the chart.

If you are willing to spend 15 days on this project and have a lot of these graphs I think it would make a lot of sense to fine-tune your own OCR and/or program some logic

---

<div class="post-metadata">

**Author:** ![christinektcastillo](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/christinektcastillo/32/354085_2.png) [@christinektcastillo](https://community.openai.com/u/christinektcastillo)\
**Post date:** [April 26, 2024, 3:42am UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/10 "2024-04-26T03:42:53Z")

</div>

![image](https://us1.discourse-cdn.com/openai1/original/4X/e/5/4/e5452a701e1ea542da9872e730b0647a2b27189c.png)

---

<div class="post-metadata">

**Author:** ![mil282](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/mil282/32/147528_2.png) [@mil282](https://community.openai.com/u/mil282)\
**Post date:** [May 14, 2024, 7:30pm UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/12 "2024-05-14T19:30:01Z")

</div>

Hello,

May I ask if you find a solution that to use Chat GPT4 to extract the curve data or even some data analysis based on that ?

---

<div class="post-metadata">

**Author:** ![mdhor](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/mdhor/32/511555_2.png) [@mdhor](https://community.openai.com/u/mdhor)\
**Post date:** [April 17, 2025, 7:16am UTC](https://community.openai.com/t/how-to-extract-data-from-a-graph-image-with-chatgpt4/608881/13 "2025-04-17T07:16:12Z")

</div>

Check out [graph2table](https://graph2table.com), it does this automatically and with good enough accuracy
