I understand that the API cannot be used to generate adult-themed content. That much is clear.
But what about using the API to help with moderation of a forum? If a user posts adult-themed content and that content needs to be flagged for moderation, will chatGPT refuse to assist? What about helping with generating tags for forum posts in an adult-themed forum? Will the adult nature of the tags prohibit the function?
If not, would using the API for moderation like this violate OpenAI policy?
Hi!
It’s not going to work.
This is what will happen: upon sending a request to the model the output will get flagged as sexual and this will be a strike against your organization.
If you send the request to the moderation endpoint first it will flag almost everything as sexual, effectively informing you that this content should not be send to the model. Besides this fact, using the moderation endpoint without using the models is not allowed.
In short: for these types of tasks you will need to use another model from another provider or go open source.
One thing the OP might consider is using an embedding model.
Embedding models have been shown to be pretty good at classification and I don’t think the OpenAI embedding endpoints are moderated in the same way chat/completion is.
So, if they have a decent set of already tagged data, then they might try using embeddings and choosing tags based on embedding similarities.
That is still not an option as the usage policies are rather strict
Don’t build tools that may be inappropriate for minors, including:
- a. Sexually explicit or suggestive content. This does not include content created for scientific or educational purposes