Should OpenAI introduce API endpoints with a built-in moderation feature?
Over the recent months, there has been a noticeable trend of many users experiencing access terminations due to unintended inappropriate inputs and/or outputs.
OpenAI already has a free solution for this called the moderation endpoint, which you can use to identify content that their usage policies prohibit and take action.
But should they introduce API endpoints for GPT-3.5T & 4 with this moderation feature built-in?
Pros:
Prevent Unintended Access Termination: With built-in safety checks, there might be fewer chances of outputs leading to access revocations.
Safety by Default: Reducing the chances of potentially harmful, misleading, or inappropriate inputs or outputs.
Ease of Implementation: Developers wouldnāt need to set up a separate moderation system or rewrite anything to use the moderation endpointās features.
Would love to hear the communityās thoughts on this. Do you think this could be a viable solution?
I would prefer that they stop having such a zero-tolerance when it comes to abuse and instead have a warning system. People can and will enter their keys in untrustworthy places and run a risk of being permanently locked out of OpenAI services. These types of policies prevent curiosity.
The janitor fiasco is a good example. Although mainly used for ā¦ Certain conversations it was and isnāt known what the prompt even was to involve a mass banning.
As someone to writes the code I like the way it is. Yet, Iād think the majority of people just want a key for other services and donāt really understand or read best practices, safeties etc etc. So itās completely unfair to give them access and expect them to know that even writing smut is a perma ban.
Itās low key madness. They seem to support BYOK for certain services but will permanently ban first-time for softcore smut. Talk about dancing on a razorās edge.
I wouldnāt like the endpoints to change. I only use the moderation endpoint for untrusted input. For personal projects itās unneeded. Mixing them together just seems to be the same, or even more work to manage. Specifically for error handling.
Sure, itās just that the Azure endpoints are āauto moderatedā as you suggest and itās a cause of problems with companies who use the API for internal, non public facing roles. Certain industrial and scientific terminology is often picked up as breaking the policy and is rejected, causing manual review processes to have to be enacted.
Also for literary applications and news use, life is often unpleasant and not allowing an AI to correct grammar or participate in the creative process at all is very restrictive, Sam Altmanās own words āI donāt want some machine telling me how to thinkā
Hereās the docs on Azure content filtering. They have some configuration around what level of moderation they perform, and you can apply for even more lax filtering.
I think if OpenAI felt the moderation endpoint was critical to chat usage, a āsimpleā solution would be to have the API send the request to moderation endpoints by default, but include a flag to skip it. Then in the documentation of the flag they can provide whatever warnings about account deletion.
Right now itās kind of mixed messaging. The docs/API doesnāt seem to push it, but thereās dire consequences if you donāt use it. A warning system with messaging about the moderation endpoint would be a welcome change.
That could also be a good solution. To be clear, Iām not suggesting that any endpoints should be replaced. Iām suggesting adding a few new ones, such as gpt-3.5-turbo-mod for gpt-3.5-turbo, to make it easier for developers to comply with OpenAIās policies.
I have a CMS and document authoring system which lets users collaborate with the AI to author documents, or to have general questions answered that the user types.
Thereās been an ongoing discussion in another extremely lengthy thread in this forum about whether the developer is EXPECTED to ALWAYS call the moderation API to check every one of these potentially unsafe queries, before calling ācompletionsā api.
None of the OpenAI policy documents answer this very simple, and very necessary question directly and clearly.
EDIT: Furthermore if the answer is āyes you shouldā then this obviously needs to be a simple āmoderate=trueā parameter in the ācompletions APIā call itself, which would obviously be defaulted to ātrueā, if they want to protect customers, which we assume they do.
Orā¦ you can just call the moderation endpoint? Why mix different concerns?
What happens when a moderation error is thrown? I need to mix my error handling now? Expect two different types of objects? It doesnāt make sense. I have one function for calling the endpoint which appropriately handles the response, and then another. Nice and clean. Modular.
Keyword is potentially. If you are handling untrusted information then it makes sense to use the moderation endpoint. If you are, for example, writing sales pitches for your line of products then you donāt need it.
Chat completion in itself is RLHFd to not do BAD stuff.
However some of the simplest and easiest things devs can do to prevent termination of their OpenAI API access because of abuse is:
Check requests with Moderation API before sending them to the models.
Pass the end-user IDs to the API when making any requests, thisāll help OpenAI know and communicate with you about users that are abusing the API via your application. This allows OpenAI to provide your team with more actionable feedback in the event that we detect any policy violations in your application.
Agreed. Itās a key for developers. Yet, the official current stance on BYOK seems to be ā¦ undefinedā¦ Weāve seen numerous developers use BYOK for whatever reason.
Although Iām not going to point fingers I have seen some support for paid applications which also support a self-hosted BYOK version. Iād say that in this case itās a great idea. Yet, Iād also say that in most cases itās just bad practices.
The problem with this approach is the unnecessarily high frequency of requests hitting the endpoint. Such an approach will unnecessarily add to compute requirements.
Imagine abusive and normal usage requests hitting the same endpoint, the decision making happening on OpenAI side whether to pass the request for generation (think about the hold time), or deny them.
Then the complexity of API response; one response for denial while another entirely different for generation.
Compare this to the scenario where only filtered requests are hitting the generation endpoint (obviously no unnecessary ones) the decision making happening on devsā systems.
The simplicity of response; devs know what kind of responses to expect from each endpoint.
+ Edit: Integrated approach will also hamper devs from bringing their own moderation system, which could be faster/better.
Communication is key.
Who should use the moderation endpoint when, how and what will happen if there are problems.
Then everybody can make their own choices.
I would leave things as they are because I think the rules are common sense.
The problem with making two API calls every time instead of one is the latency, and degrading performance for no reason.
I can see use cases where developers might have a need to call the āmoderationā endpoint standalone, with no intent to RUN the query if it passes moderation, but just because thatās true doesnāt mean that 100% of all of the most common usage patterns of the API should always require two back to back HTTP requests.
The obvious solution is a āmoderate=trueā flag on the completions endpoint, that defaults to true.
Yes indeed, the additional HTTP call adds latency.
I think one of the most common use cases is āchat botsā where apps take raw input from end users, and so 100% of those calls would need to be sent thru moderation (according to many peoples interpretations of the vague policy docs), and so if thatās the case the rational choice for API design is to have moderation built into the ācompletionsā call, as an option that defaults to true.
Iād like to say to the API, āblock this if it violates OpenAI policyā, and donāt ding my API key. So maybe the more applicable parameter name is āding=falseā. lol.
I can agree with that, I think itās worth delving a bit into this.
This is absolutely true, letās remember that the moderation endpoint is an AI classifier. Itās probably not as resource-intensive as GPT, but it will still require GPUs, and those are currently in short supply.
Although I think itās worth mentioning that OpenAI is already passing the input, output, and generated title on ChatGPT through the moderation endpoint, so they do already know how to successfully put those cogs together.
