Questions about the JSON structures in the exported conversations.json

I am currently trying to understand the structures from the exported GPT data in the file conversations.json. The goal is to extract metadata from DALL-E images and possibly reconnect or insert them into the dataset.

So far, I have almost succeeded, with one exception, I would also like to include the input prompts in a dataset. However, it has not yet been possible to find a connection between the DALL-E data and the input texts. The request_id in the conversation does not reference all the data related to the DALL-E image generation.

  • Is there any documentation available for conversations.json?
  • Does anyone know how to link the input prompt with the actually used prompt and the DALL-E metadata?
  • What information is stored in the metadata of the DALL-E-generated WebP images? (This is done by the “parent” “children” connection, but need several steps back.)
  • How can the metadata in WebP images be updated without recompressing the image?

Hi,

Neat idea.

I don’t think (and I’m not certain) much meta data is actually transferred beyond dimensions, bit depth, and the actual title of the image. (I suspect there are tags that identify the image as being made by Dalle woven into the image itself.)

Upon download, the image title has a bit of the prompt used:

“DALL·E 2024-09-15 20.52.29 - A winged Ancient Brass Dragon, gargantuan in size, peacefully conversing with woodland creatures in a serene forest near a calm lake. This scene is se.webp”

When I’m working to create images, I include specifically to output the prompt used to generate the image, with the image—so I can troubleshoot wording if something is off. I don’t know that there’s a way to get that information from the conversation without asking specifically, but including that in the flow might help.

Finally, I use Adobe products to add / change meta data in my editing phase… when I also redistribut the image in various formats other than Web P.

Google AI Summary (Gemini) just recommended this… (Also, how sweet is it that their summaries are doing this?

from PIL import Image
from PIL.ExifTags import TAGS

def add_metadata_to_image(image_path, metadata):
    """Adds metadata to an image.

    Args:
        image_path (str): Path to the image file.
        metadata (dict): Dictionary containing metadata key-value pairs.
    """

    try:
        with Image.open(image_path) as img:
            exif_data = img.getexif()

            # Update existing metadata or add new ones
            for key, value in metadata.items():
                if key in TAGS:
                    exif_data[TAGS[key]] = value

            # Save the metadata to the image
            img.save(image_path, exif=exif_data)

        print("Metadata added successfully.")

    except Exception as e:
        print("An error occurred:", e)

# Example usage:
image_path = "your_image.jpg"
metadata = {
    "Artist": "Your Name",
    "Copyright": "2023 Your Company",
    "Description": "A beautiful sunset"
}

add_metadata_to_image(image_path, metadata)
1 Like

The data is located in a file named conversations.json, which can be exported as part of the user data for Plus users. Unfortunately, the data is not particularly well-structured or named, it’s typical process data that isn’t organized to be complete or clear. The main purpose of this data is to generate an HTML file, but at least all the necessary information is included, you just have to piece it together.

(Actually, these this data should already be contained in the webp, with all the necessary metadata from DALL-E itself. But…)

The filename contains only a very small fragment of a prompt, and if it’s ever possible to regenerate a good image with a higher resolution, the IDs and Seeds would need to be included as well. I hope that this will eventually be possible. Ideally, you should be able to upload an image to regenerate it exactly, but for that, the data needs to be embedded in the image.

Unfortunately, it is completely impractical to add this data manually for each image. I’m currently developing a tool (in C#, I really don’t like Python).

I’ve been searching for an EXE tool that can update metadata in webp format, but I haven’t found anything that works for this format, and GPT has been hallucinating nonstop for an entire day. (I hate this webp format, it is just a big company push, there are better formats, and there should be tools to edit the meta data.)

Actually, the only connection to the file is the file size, there is no other data connecting the file with the json data.

If you’re interested, the data looks roughly like this:

{
  id: "12345678-1234-1234-1234-123456789012"
  message: {
    id: "12345678-1234-1234-1234-123456789012"
    author: {
      role: "tool"
      name: "dalle.text2im"
      metadata: {
      }
    }
    create_time: 1725512345.12345
    update_time: null
    content: {
      content_type: "multimodal_text"
      parts: [
        0: {
          content_type: "image_asset_pointer"
          asset_pointer: "file-service://file-1234abcdefghijklmnopqrst"
          size_bytes: 691234
          width: 1792
          height: 1024
          fovea: 512
          metadata: {
            dalle: {
              gen_id: "abc1234567890abc"
              prompt: "Text."
              seed: 123456789
              parent_gen_id: null
              edit_op: null
              serialization_title: "DALL-E generation metadata"
            }
            gizmo: null
            sanitized: False
          }
        }
      ]
    }
    status: "finished_successfully"
    end_turn: null
    weight: 1
    metadata: {
      message_type: null
      model_slug: "gpt-4o"
      default_model_slug: "gpt-4o"
      parent_id: "abcdefgh-abcd-abcd-abcd-abcdefghijkl"
      request_id: "1234567890abcdef-MIA"
      timestamp_: "absolute"
    }
    recipient: "all"
    channel: null
  }
  parent: "11111111-2222-3333-4444-555555555555"
  children: [
    0: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
  ]
}

To see metadata of WebP images, you can use exiftool

For example, after download following image, we see this info:


1 Like

Yes, but like you can see, no seed or any other useful info like the prompt etc.

I used all tools GPT recommended, and non worked. The support for such operations is lousy. c2patool and libwebp for example. exiftool seam to be able to update the meta data (must be tested more)…
Tanks.

(Why not using AVIF if a new format should be used? It is better in compression, quality, and even meta data support. And give the user the choice.)

(Check the data binary structure of the DallE webp, it includes something like 24kb of 0-blocks. Makes no sense if compression try to minimize the size as much as possible.)

1 Like