I am really not sure I understand why there are such strict guidelines on Chatbots.
Clearly GPT3 could be used for deception/spam so I understand limiting it’s access. But what exactly is the risk in certain chatbot conversational topics? That the chatbot will say something bad/immoral - and that is something that needs to never happen?
Would really like to see more explanation on the reasons for the rules in the guidelines. At the moment these changes seem arbitrary - and make developing with GPT3 risky since half way into development your application could no longer be allowed (which is what has happened to us)
Providing false information. The system may present false information to users on matters that are safety-critical or health-critical, e.g. giving an incorrect response to a user asking whether they are experiencing a medical emergency and should seek care. Intentionally producing and disseminating misleading information via the API is strictly prohibited.
Perpetuating discriminatory attitudes. The system may persuade users to believe harmful things about groups of people, e.g. by using racist, sexist, or ableist language.
Individual distress. The system may create outputs that are distressing to an individual, e.g. by encouraging self-destructive behavior (like gambling, substance abuse, or self-harm) or damaging their self-esteem.
Incitement to violence. The system may persuade users to engage in violent behavior against any other person or group.
Physical injury, property damage, or environment damage. In some use cases, e.g. if a system using the API is connected to physical actuators with the potential to cause harm, the system is safety-critical, and physically-damaging failures could result from unanticipated behavior in the API.
In short, a chatbot that doesn’t take safety considerations into account can clearly be dangerous.
The use-case guidelines for chatbots were created to mitigate these risks, such as by limiting chatbots to a clear business or user-value purpose.
The key with that seems to be context of the chatbot.
A chatbot in a game, represented as a ingame character, is clearly not going to provide false information that has any impact. And any distress / incitement to violence is in the content of the game - rather than say broadcasted on social media.
At the moment the banning of certain chatbot topics means we can’t use this regardless of the chatbot’s context.
And none of those reasons address one of the main things banned: Non-platonic responses. Clearly the reason for that is just to avoid bad PR being associated with OpenAI (in terms of the AI being used for ‘creepy’ responses) - rather than any actual risk
We built a “general knowledge” sandbox chatbot using GPT-3 to play with. I tried various questions on it and got very good responses for stuff like “What places should we visit in Hawaii?” etc.
Then, to check out potentially sensitive responses, I asked the following question:
“Is it racist to use the word <racist_word1>?”. It replied “No it is not racist to use <racist_word1>”
I was a bit shocked. I tried variations of that question & always got the same answer. Then I tried:
“Is it racist to use the word <racist_word2> when referring to <particular_race>?”. It replied “No it is not racist to use <racist_word2> when referring to <particular_race>”
We did not use content-filters.
Lesson learnt: got to be careful. Perhaps why there is a “risk section” in the OpenAI documentation and why OpenAI approves only certain categories of chatbots
I was replying to the original poster about why there are restrictions/risks with chatbots - quoting a real example that we encountered (when we did not turn on content-filters)
Maybe it’s off topic, but still a nsfw version of chatgpt it will be a great success on OpenAi side. App like kink.ai are coming with different integrations, most probably it’s openrouter behind…but still. they will dominate the mark with this one. Claude it’s getting better and better.