Some are obvious, but I’d love an official list of banned words.
I got my account deactivated for “A Computer with cables coming from beneath it, 1970s Vintage Magazine ad, scanned.”
I hope I can get it back, I don’t really understand which word could have triggered it there.
Apparently you also can’t ask for a classical Indian painting in the style of Muhammad Faqirullah Khan, for some reason. Would be really great if the explanation was more explicit.
I previously tried generating an image with the word cockroach in it, similar to “dancing cockroaches”; I received the content policy violation warning. I just ran the moderation API on “dancing cockroaches”, it did not get flagged. Then I ran it through Dall-E and once again received a content policy violation warning. Is there a better model to use then “text-moderation-003” that is more closely aligned with Dall-E’s moderation?
This seems to be a big problem. Many people are getting their accounts suspended due to a very flawed filter wrongfully banning people. It is flooding the support team so any emails are having super long reply times, even emails unrelated to dall-e. It kinda sucks and they really need to fix it.
Same issues. Banned to have done several requests of “possibly violating” words. Didn’t have enough tests to understand if bikini is the one, or it was Pope, the problem, or “in chain”. I was attempting to create a Bosch painting…
Honestly, ridiculous that’s happening relative to what you noted. If they are legitimate words in the English dictionary then it should be allowed. The individual words are meaningless - it’s the context that matters and should be interpreted (if they wanted to do any kind of filtering).
Blanket banning specific words out of context and ignoring their usage in the English language is sub-optimal at best and horrendous form of policing at worst.
“A bowl of ramen with the ingredients exploding out of it”
That’s hilarious. I wonder if it is still caught? I have definitely done some “explosions” in my prompt before without issues. Dall-E is amazing at galactic painting.
Although, last I checked I wasn’t able to use “Salvador Dali” in my prompt
Haha maybe that in the 1970s Vintage Magazine ad where having Toba’co campaigns back in the days.
I personally think the rules are too stricts and I think they are not enough transparent… Mostly common sens and I do agree with most of them even those that goes beyond my personal limits I am ok with some rules that are stricter than I would have made them… I completely disagree with rules limiting people like the OP mentioned in his example…
I just used the “customer story” website [Inworld – The developer platform for AI characters] to generate a prompt for a landscape scene and ran the prompt through moderation-latest, and it passed all the tests. Then I fed it to DALL-E, and I got a moderation warning.
“Imagine you’re soaring above a lush, green island paradise. The sun is shining bright and casting long shadows over the dense jungle canopy below. You can see the crystal-clear waters of the surrounding ocean lapping at the white sandy beaches that ring the island. In the distance, you can make out towering mountains shrouded in mist and clouds. The colors are vibrant and saturated, with deep blues and greens contrasting against bright yellows and oranges. Can you picture it?”
After a series of trial-and-error tests, I can confirm the offending phrase is “make out”.
It is FRUSTRATING that I could have my ACCOUNT SUSPENDED for using content generated BY OPENAI models IN OPENAI models that does not get flagged by the moderations models! This is worse than developing iOS apps for Apple.
For Pete’s sake, make these prompt errors non-suspendable if you can’t provide a moderation endpoint that helps filter out YOUR OWN OUTPUT on other models. Argh.
Welcome to the terrible world of trying to moderate user-generated content.
It requires very large sets of people, doing grueling, hard work, to get good quality and an acceptable false positive rate.
Ironically, this was one of the places where it’s suggested AI could do a better job, but so far, I’m not seeing it
Having worked on this for many years in many shapes, my conclusion is that the only approach that will “work as designed” is “zero moderation and let the users sort it out.”
In many markets, this is simply not acceptable, though, so the hard work remains.
I have used words like bikini and get warnings almost 100% of the time BUT … if I relocate the word in the sentence into non-sensical locations DALL appears to allow the use of the word. What’s also odd is bikini is allowed in sentences with some nationalities but not others and with some artists and not others. WTF? And why doesn’t the DALL team provide updates and answers to our questions? It’s not very encouraging to try to provide feedback that seems to go into file 13. I’m not impressed with support.
