# Vision API - "Image not allowed by our safety system"

**URL:** <https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147>\
**Category:** Bugs\
**Tags:** gpt-4-vision\
**Created:** [March 12, 2024, 1:59pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147 "2024-03-12T13:59:06Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![kristsdarzins](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kristsdarzins/32/331454_2.png) [@kristsdarzins](https://community.openai.com/u/kristsdarzins)\
**Post date:** [March 12, 2024, 1:59pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/1 "2024-03-12T13:59:06Z")

</div>

Hi all! I’m experiencing trouble with multiple images, here’s one example - an image of a sign on the wall. However, the API responds with “Your input image may contain content that is not allowed by our safety system”. This isn’t the only image that, despite appearing normal, gets flagged for some reason. Can anyone explain why this is happening and if there’s a fix? Alongside this, I’m asking ChatGPT to describe what’s in the image. Thanks!

 ![016_WALL_SIGN_DARK_GLOSSY](https://us1.discourse-cdn.com/openai1/original/4X/5/8/d/58d2f47538e56677369b5567647a9a883f45da26.jpeg)

---

<div class="post-metadata">

**Author:** ![jck](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jck/32/318603_2.png) [@jck](https://community.openai.com/u/jck)\
**Post date:** [March 12, 2024, 2:08pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/2 "2024-03-12T14:08:17Z")

</div>

My guess is that this is part of their captcha defence. If it allows for detecting numbers, one could use it to defeat captchas. Annoying when you’re just trying to use it for legit purposes, though.

---

<div class="post-metadata">

**Author:** ![kristsdarzins](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kristsdarzins/32/331454_2.png) [@kristsdarzins](https://community.openai.com/u/kristsdarzins)\
**Post date:** [March 12, 2024, 2:12pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/3 "2024-03-12T14:12:35Z")

</div>

That would be kind of silly because there are a lot of images with numbers out there, haha. I hope this is just a temporary issue. By the way, here’s another image that the API is having trouble describing:

 ![MM_BusinessCard_BC-H-04](https://us1.discourse-cdn.com/openai1/original/4X/d/8/5/d85a2e7eb20bd2a1efca5ddbe5d3b3051032d64a.jpeg)

---

<div class="post-metadata">

**Author:** ![wclayf](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/wclayf/32/184514_2.png) [@wclayf](https://community.openai.com/u/wclayf)\
**Post date:** [March 12, 2024, 8:07pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/4 "2024-03-12T20:07:14Z")

</div>

OpenAI has disabled most OCR type functionality, in my opinion. They give various reasons, at various times, as the _reason_ for doing so. Any attempts to use OCR for applications may work sometimes, and fail sometimes. Definitely not reliable enough to use for an application.

---

<div class="post-metadata">

**Author:** ![kristsdarzins](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kristsdarzins/32/331454_2.png) [@kristsdarzins](https://community.openai.com/u/kristsdarzins)\
**Post date:** [March 12, 2024, 8:28pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/5 "2024-03-12T20:28:19Z")

</div>

I wouldn’t be so critical, but there are indeed some restrictions.

Over the last 7 days, I’ve requested descriptions for about 10,000 images through the API, and it’s generally been working well.  
Though a small percentage of images are getting declined.

I’m trying to understand the issue, as some of these images are important too. Interestingly, most of the images included texts too, which the AI was able to read without any problems. The rejections seem very random.

---

<div class="post-metadata">

**Author:** ![anon22939549](https://avatars.discourse-cdn.com/v4/letter/a/f14d63/32.png) [@anon22939549](https://community.openai.com/u/anon22939549)\
**Post date:** [March 12, 2024, 8:41pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/6 "2024-03-12T20:41:24Z")

</div>

One thing which can help is to modify the image slightly to make it look less like a CAPTCHA.

I discovered this as a side-effect of using “set-of-marks” prompting with the vision model.

> **[Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://arxiv.org/abs/2310.11441)**
>
> We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V. As illustrated in Fig. 1 (right), we employ off-the-shelf interactive segmentation models, such...

 ![Screenshot from 2024-03-12 13-21-57](https://us1.discourse-cdn.com/openai1/original/4X/a/7/7/a776653e9d235d0bce7019044825d8acd631ccde.png)

 ![Screenshot from 2024-03-12 13-45-38](https://us1.discourse-cdn.com/openai1/original/4X/2/6/1/261518c63dfd379df5913b00825c59cfb0328d74.jpeg)

> **overlay\_grid.py**
>
> ```python
> import argparse
> from PIL import Image, ImageDraw, ImageFont
> import os
> 
> def draw_labeled_grid(image_path, horizontal, vertical, font_size):
> print(image_path)
> with Image.open(image_path) as img:
> draw = ImageDraw.Draw(img)
> width, height = img.size
> cell_width = width / horizontal
> cell_height = height / vertical
> 
> font_path = '/usr/share/fonts/truetype/noto/NotoMono-Regular.ttf'
> try:
> font = ImageFont.truetype(font_path, font_size)
> except IOError:
> font = ImageFont.load_default()
> 
> text_padding = font_size / 8  
> line_color = 'blue'
> line_width = 4
> for h in range(horizontal + 1):
> draw.line((h * cell_width, 0, h * cell_width, height), fill = line_color, width = line_width)
> for v in range(vertical + 1):
> draw.line((0, v * cell_height, width, v * cell_height), fill = line_color, width = line_width)
>         
> num_digits = len(str(horizontal * vertical))
> for h in range(horizontal):
> for v in range(vertical):
> label = str(h * vertical + v + 1).zfill(num_digits)
> text_x = h * cell_width + text_padding
> text_y = v * cell_height + text_padding
> 
> draw.text((text_x, text_y), label, fill = line_color, font = font)
> 
> output_path = f"labeled_grid_{os.path.basename(image_path)}"
> print(output_path)
> img.save(output_path)
> return output_path
> 
> if __name__ == " __main__":
> parser = argparse.ArgumentParser(description='Draw a labeled grid on an image.')
> parser.add_argument('image_path', type=str, help='The path to the image file.')
> parser.add_argument('horizontal', type=int, nargs='?', default=3, help='The number of horizontal cells.')
> parser.add_argument('vertical', type=int, nargs='?', default=3, help='The number of vertical cells.')
> parser.add_argument('font_size', type=int, nargs='?', default=48, help='The font size for the labels.')
> 
> args = parser.parse_args()
> 
> draw_labeled_grid(args.image_path, args.horizontal, args.vertical, args.font_size)
> 
> ```

---

<div class="post-metadata">

**Author:** ![kristsdarzins](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kristsdarzins/32/331454_2.png) [@kristsdarzins](https://community.openai.com/u/kristsdarzins)\
**Post date:** [March 12, 2024, 8:50pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/8 "2024-03-12T20:50:55Z")

</div>

This is really interesting, I’ll dig deeper, thanks!

We’ve developed a WordPress plugin that automates the writing of attachment metadata, so we’ll see if we can reduce the number of skipped images with a similar approach.

---

<div class="post-metadata">

**Author:** ![wclayf](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/wclayf/32/184514_2.png) [@wclayf](https://community.openai.com/u/wclayf)\
**Post date:** [March 12, 2024, 8:53pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/9 "2024-03-12T20:53:12Z")

</div>

That wasn’t being critical, but simply accurate, to help members of this community not waste time on something that will fail in production.

Mostly it’s “business related” information that OpenAI will refuse to OCR, like people’s names, addresses, emails, phone numbers, company names, etc. So as long as your use case doesn’t involve business info you’ll be fine, …unless/until OpenAI changes their mind and censors your use case as well.

---

<div class="post-metadata">

**Author:** ![kristsdarzins](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kristsdarzins/32/331454_2.png) [@kristsdarzins](https://community.openai.com/u/kristsdarzins)\
**Post date:** [March 12, 2024, 8:56pm UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/10 "2024-03-12T20:56:57Z")

</div>

This is very true, yes.  
We created a WordPress plugin that writes image titles, ALTs, etc., so it really uses an API for the very basic feature it should do - see what’s in the image and understand it.

---

<div class="post-metadata">

**Author:** ![Craign](https://avatars.discourse-cdn.com/v4/letter/c/ad7895/32.png) [@Craign](https://community.openai.com/u/Craign)\
**Post date:** [May 29, 2024, 3:41am UTC](https://community.openai.com/t/vision-api-image-not-allowed-by-our-safety-system/679147/11 "2024-05-29T03:41:36Z")

</div>

I’m getting the same error message uploading a picture of seeds on a plain background for the model to count