OpenAI seems to have developed a robust framework for their Bug Bounty Program, but the process for reporting other safety-related issues is somewhat ambiguous.
I get that it’s challenging to respond to every inquiry, but issues related to model security should take precedence. It’s not too hard to implement standardized responses. Currently, if you submit a comprehensive report about a security vulnerability, chances are you won’t hear back.
If all submissions via the https://openai.com/form/model-behavior-feedback portal are reviewed, that means there’s an opportunity for feedback to the person who filed the report. Here are some potential categories for classifying the reports:
- Duplicate report (issue already flagged by another user or internally)
- Inadequate report (feedback is poorly organized, due either to a lack of clarity or coherence)
- Inaccurate report (information is either factually incorrect or not actually an issue)
- Third-party report (the problem lies with a different entity or system, not OpenAI)
- Resolved report (issue has already been addressed in recent updates)
- …
After evaluating a report, the team reviewer can certainly categorize its usefulness. This can then be communicated back to the report’s author automatically.
This would not only create a productive learning cycle but also generate valuable statistics, which could be used in the future to enhance the quality of reports showcased on a dedicated webpage.