My name is Andrea from Riavvio.ai, and this is my inaugural interaction with the community. I kindly ask for your patience and understanding should there be any inadvertent errors in my approach.
I am currently engaged in exploring the GPT-4 Vision model, specifically for analyzing user-uploaded images to deter the spread of unsuitable content. The methodology I’ve adopted is based on the examples provided at gp4v examples.
However, I’ve encountered a challenge with the model’s application. While processing an image, I encountered the following error:
.../lib/python3.12/site-packages/openai/_base_client.py", line 877, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError:
Error code: 400 -
{'error': {'message': 'Your input image may contain content that is not allowed by our safety system.',
'type': 'invalid_request_error', 'param': None, 'code': 'content_policy_violation'}}
I acknowledge and respect OpenAI’s content policy, which aims to maintain the safety and appropriateness of the content processed by its models. It appears that the second image I tested may have inadvertently contravened this policy.
I seek the community’s guidance and insight on several aspects to ensure my application aligns with OpenAI’s standards:
Is the GPT-4 Vision model an appropriate tool for filtering potentially offensive user-generated content in a manner that is both safe and in compliance with OpenAI’s policies?
In the event that the error stemmed from the image’s nature, could you provide specific content guidelines or restrictions to assist in aligning my usage with OpenAI’s policy requirements?
Are there implications for my account if I conduct extensive image content reviews to uphold the safety and integrity of my platform?
I am committed to adhering to OpenAI’s guidelines and policies and would greatly appreciate your advice and support in this endeavor.
Thank you for your time and consideration. I eagerly await your response.
The vision system is not intended in it’s current form for image moderation, there may be such services offered in the future as AI moderation is being explored by the OpenAI team for commercial applications. I would suggest reaching out to sales@openai.com and possibly leaving your contact details on the supoprt bot in the bottom right corner of help.openai.com
you might want to try llava instead. it has zero safety system whatsoever, therefore, you can try it for an image content moderation support. replicate.comm will help you, yet you can use whatever provider to run the model.
We often come across the ‘content_policy_violation’ error for content that doesn’t seem to violate any known policies or be offensive. For instance, we received this error for a picture showing only a Razer brand computer mouse on a mouse pad. However, when we used the same image with Gemini Pro, we didn’t encounter any problems. So, despite my reservations about discussing this on an OpenAI forum, I will say our approach now is to use Gemini Pro Vision as a fallback upon this error, and if that doesn’t work, we turn to LLavA as our final recourse.