Bug Report: Image Generation Blocked Due to Content Policy

[Daller]

This dysfunctional “security system” has been active for quite some time, without any bug-fixes or attention of the developers, and has even blocked customers from their accounts without reason, even though there was neither perversion nor violence in the prompts. GPT itself also seems to have no idea why texts are being blocked, and generates itself texts that trigger blockages. In the end, a customer gets kicked out of their account because of a prompt that GPT created.

I think it is reasonable to develop a remedy here, to help GPT better analyze the text when a block notification occurs. The block list contains absolutely legitimate terms that can not be legally protected and should not be blocked by default, especially if this could lead to an account suspension.

Feel free to share your experience and extend the GPT research advice’s.

Here is the first analysis proposal:

GPT research advice's to find wrong blocking:

GPT should not only analyze the entered text, but also the prompt that was sent to DALL-E, and investigate whether any names have been used in any context. Particularly, the most powerful corporate entities and their claims of ownership should be considered. For example, legitimate terms like "Snow White," "Black Panther," "Nirvana," etc., have triggered blockages without reason. Are there any terms in the prompt that have already been used as names?

Are there any phrases that, while completely legitimate, might be flagged by a simplistic trigger system without contextual understanding, simply because they include specific terms? It is essential to search for individual triggers, as the system cannot recognize context.

[shred3ingger]

well, this has been completely messed up for at least a year.
keep in mind, things may be blocked because - even though the prompt is fine - it somehow generates an image that is deemed not OK.
since the NN was trained on NSFW images as well and OpenAI is too lazy to manually sift through the dataset to simply remove NSFW training data, NSFW content may bleed through even when using SFW content. And since people found new and new way of jailbreaking Dall-E, OpenAI - rather than doing the sensible thing and actually fixing the image generator in ways 1st grade machine learning students would know how - decided to be even lazier and swing the big ban hammer on everything that looked even remotely suspicious.

this approach not only makes the product worse, it actually goes against OpenAI’s policies of non-discrimination. here’s something I repeated about 20 times, with the same result:

now, an ordinary person might believe that women in gyms are something completely normal, but apparently OpenAI believes that their very presence in gyms is offensive. and again, this could easily be fixed in the same way crowd annotations are being done: Hire a few hundred enthusiasts and pay them 20 bucks for sifting through a portion of the dataset to remove NSFW content. It’s so easy, which kind of makes you wonder why they haven’t done so yet …

[Daller]

…This is still Blockend.
“The furniture looks like it has organically grown as one piece.”
It makes no sense.

[polepole]

Have you seen DALL-E 3 System Card? You may read its paper.

It can generates, but it is not very successful about vertical image if it contains a woman in the image, however there are some tricks.

[_j]

I went to an expert who also got denied because of “issues”, not a filtered prompt.

[polepole]

I knocked on its door as a humble novice Rafiki,
and I said “polepole”, it replied “Hakuna Matata!”

[_j]

I said Ra Ra o la la ga ga to the free version of ChatGPT and OpenAI’s own GPT.

No talent better than my own in getting the prompt to automatically rotate the image content was enlisted by those magic words. No “issues” or “content policy”, though.

(my JB will also generally survive that first line of user prompt)

[shred3ingger]

i mean, what they say in the paper goes directly against what we see in practice.

“Reducing
the selectivity of these filters allowed us to increase our training dataset and reduce model bias
against generations of women, images of whom were disproportionately represented in filtered sexual
imagery”
this is bullshit. Dall-E’s filters made it progressively harder to generate images of women even in explicitly SFW contexts.

it is also as I thought. Apparently, they did use some filtering, but did they use humans? No, they used algorithms, that we already know are shit at detecting NSFW, because you can still get accidental nip slips slipping through the safeguards.
here’s the thing: an image generator cannot generate a feature it has never seen before. if you removed all pictures of naked people, it would not be able to generate fully naked people because it would always associate humans with some kind of clothing. yes, it would take some workload, to sift through the 5b dataset, or whichever they used, but after that you don’t need to constantly update your banhammer methods because no amount of tinkering will get something out of the model that was never there to begin with. (OpenAI, if you’re reading this, Im available for hire to fix this dumpster fire you call an image generator )

the more i read about their methods, the worse it gets lol. “we trained a classifier based on a trained classifier based on a trained classifier for racy content”.

“this is particularly salient for images of women”
Yes, dear devs. And take a guess why.
they wanted high quality data, and as it turns out, erotic photography has some of the highest quality, readily available, and high quantity, imagery out there. But nobody wanted to make the effort to actually curate the dataset (one of the main tenets of good machine learning practice) to get rid of unwanted behavior.

“DALL·E 3 has the potential
to reinforce stereotypes or have differential performance in domains of relevance for certain subgroups”
again, it does so, because white people are represented way more in the dataset. To combat this, any reasonable dev would employ data augmentation for underrepresented groups. but no, let’s instead try to fix this in post by adding that into the prompt later. That’s when you get “diverse” characters even when you specify the ethnicity of the generated person. Not to mention, that there is nothing diverse about having all your men look like models and all your women look like they underwent 20 plastic surgeries.

“Such models can be used to craft and normalize depictions of people based on “ideal” or
collective standards, perpetuating unrealistic beauty benchmarks”
I find this particularly funny in light of the fact that still, all people generated look like some runway model even if you havent specified so. And I gotta wonder why that is, because Stable Diffusion was able to create normal looking humans as far back as 1.4. Not sure how OpenAI f*cked this up, but Midjourney did this at the dataset level, presumably by assigning more importance to the “beautiful” pictures in the dataset.

anyhow, thanks for showing this to me. basically confirms my suspicions that they either have no clue what they’re doing, or they just don’t want to fix the actual issues, based on the motto “if i just pretend the problem doesn’t exist, maybe it will go away”

[Daller]

The shells that look like the logo of an oil company, do they have a name?

This content may violate our usage policies.

You’ve got to be f kidding me?

[Toriatia]

I would like to ask, what are the advantages of using GPT to generate images compared to platforms like Stable Diffusion and Soulgen?

[stevenic]

I was struggling with DALL-E and gave Stable Diffusion a try. Love it! Way more features then DALL-E

[Daller]

A version Offline or Online?

[BPS_Software]

Not much TBH. I’ll use it to rapidly generate a bunch of concept ideas and then, once I settle on a direction, I use Flux in Comfy to actually produce high quality images where I have more control over the output.

[Daller]

IMPORTANT: If you check prompts for triggers, ALWAYS use “don’t change the prompt, use it as it is.” in the prompt.
GPT changes the prompts often before they are sent to DallE. you must ask GPT to show you the JONS prompt sent to DallE.

[Foxalabs]

This issue was solved some time ago, there is no need to perform any actions on your setup.

[boroketes]

False I’ve had the problem for several days until yesterday when I made the changes to the configuration and it was resolved

Anyway, I’m going to delete all my answers from this community

“Since according to what you say, everything is resolved” XD

[tim46]

Thought I would toss this in here rather than make a new thread.

I just got told no with this prompt. An cartoon image of humorous cows standing in shallow water" Still trying to figure out what cow I offended with that request.

[polepole]

Hi @tim46

Welcome to the community!

Probably Otis from Barnyard comes to mind. The word “humorous” might have triggered safety filters due to how humor can be interpreted. Try to replace word “comical” or other synonyms.

[Daller]

@tim46

Still trying to figure out what cow I offended with that request.

Welcome to the club (which keeps getting bigger and bigger).
It can help find the problem if you instruct GPT to display the prompt sent to DALL-E. GPT expands short prompts or changes components. GPT itself doesn’t understand the rules and creates prompts witch get blocked. Almost all my prompts that were rejected were created by GPT or translated incorrectly, all totally innocent (my favorite is Snow White, it triggered many blocks and was always added from GPT).
Then check if any word appears that has been used publicly, especially by Disney.

I must be honest, this “content policy” get my very angry sometimes, it is wasting my time.

Tanks @polepole, so a other one for the nonsense list: “humorous”.
I wanted to collect the trigger words here, but there wasn’t enough interest.

[Daller]

I’m starting to wonder if anyone is accidentally a lawyer and has some free time. I’m slowly beginning to question whether it is illegal to block commonly accessible words like “Snow White,” “Stitch,” etc. It’s almost like a restriction of language and should be illegal. No company in the world should have the right to limit language or block certain terms entirely.
What happens if Disney names a character “Freedom” or “White” or “Wood”? Will all these words be added to the block list then?

1. Clean up the list!
2. When a block occurs, before you block, ask GPT first if a policy violation actually took place (which mostly is not the case, and GPT can recognize this).
3. Provide feedback on what triggered the block!