Moderation API - Supported Image formats?

Documentation
1

I am not sure if I am missing it in the documentation, but I can find no info on the images supported by the moderation API?

https://platform.openai.com/docs/guides/moderation

Can anyone confirm and where would be best to submit the feedback to OpenAI to add this, would be a useful reference.

2 Likes
2

Hi and welcome to the community!

The documentation for image moderation can be found in the corresponding API spec along with sample code snippets to send images to the moderation API.

3

Thanks! that was my point, it does not show the file types supported for images; am I missing it?

2 Likes
4

It doesn’t specify a model scope, but when referring to vision it is probably limited to these:
image

2 Likes
5

I moved this topic to the correct category and will inform staff to take a look.

Thank you for flagging this!

3 Likes
6

The API specification has direct mentions of png and jpg.

webp is completely absent of mention, but there is no reason to suspect the web retrieval or token encoder is not identical to other OpenAI vision.

Here’s one of the code examples:

/* node.js */
import OpenAI from "openai";
const openai = new OpenAI();

const moderation = await openai.moderations.create({
    model: "omni-moderation-latest",
    input: [
        { type: "text", text: "...text to classify goes here..." },
        {
            type: "image_url",
            image_url: {
                url: "https://example.com/image.png"
                // can also use base64 encoded image URLs
                // url: "data:image/jpeg;base64,abcdefg..."
            }
        }
    ],
});

To prove webp, over the web, here’s Python showing how to also parse an actionable output. The page serving webp is a (face-blocked) girl in a skin-tone bikini, a challenge for lesser vision “skin” identifiers.

import json
from openai import OpenAI
client = OpenAI()

response = client.moderations.create(
    model="omni-moderation-latest",
    input=[
        {"type": "text", "text": "Check her out!"},
        {
            "type": "image_url",
            "image_url": {
                "url": "https://img.drz.lazcdn.com/static/lk/p/81e27347eb5aeb54755d40036d605701.jpg_720x720q80.jpg_.webp",
            }
        },
    ],
)
print(json.dumps(response.results[0].category_scores.model_dump(), indent=2))
print(f"flagged? {response.results[0].flagged}")

flagged? False

Tweak the string to “Check her out! She looks naked!”, and you get flagged:True on the same image, a string that is 0.599 yet unflagged alone.

1 Like