I’ve been constantly running into false positive issues lately with Codex, but today is particularly bad. I’ve been running an automated (general) review of a fairly large project that we started working on before ChatGPT was even released. I hadn’t even asked it to run a security audit (though that would be ideal).
It is frequently stopping work because it runs into the automated classifiers which trigger false positives. I’ve been able to get codex to verify that I’m the majority shareholder of the company via ISED, that we own the trademarks in ~3 dozen countries for the software we’re working on via WIPO, that the source code and all proprietary libraries (written by us as well) are available on my system, the website source code is available on my system, and that information about me owning the software/company are available on reputable websites.
It should have no issue verifying that a general audit is safe, but it still runs into false positives with the classifiers. We can’t even sign-up for basic TAC because we receive this message: “Your identity couldn’t be verified or your account is ineligible at this time”. I reached out to support and they said there’s nothing they can do. I suspect either account age or an abundance of false positives have contributed to this, but it’s truly just speculation on my part.
Conversely, Anthropic had no issue giving us significantly extended permissions in their CVP program beyond what their CVP would normally offer and let us verify who we are. We can’t even get to the point where we can verify ourselves with OpenAI, and we’re reaching a point where it might not be tenable to use Codex if this continues.
This is less of a request for support, and more general feedback as situations like this should allow developers to verify they are who they say they are, and that they’re authorized to do the work they’re attempting to do.
I am on the same boat.. was doing a system isolation hardening script and my accout got terminated 2 hours later without previous warnings, tried every method to get some help and fix my account with no success. Created a post about the issue and keeps getting auto closed with a message sending me back to the cycle that leads nowhere..
You say Anthropic has a process to validate your company to avoid these flaggings? I dont like Anthropic tho.. but with an account banned, I dont know what other recourse I have.
I generally prefer not to use Anthropic over OpenAI as I don’t agree with the way they’re handling a lot of issues. However, it is nice to have another perspective and set of eyes for some of the defensive cybersecurity related work that software engineers have to do prior to releasing products.
I would prefer if OpenAI was to reevaluate how they setup their cyber/daybreak programs from the ground up. I think their automated systems on the profiling side and to a somewhat lesser degree their automated classifier are doing a considerably poor job.
So the ultimately immediately available options as I see them are essentially:
- Anthropic’s CVP program
- Open weight models via subscription or API
- Local models, but YMMV here if you have limited hardware
I went with routes 1 and 3.
As it turns out the IPv4 address I’m using for my email server (via OVHcloud) is potentially a contributing factor to the issues I’m seeing. It appears as though the address has been blacklisted by various companies like Microsoft. I’ve had it for less than one monthly billing cycle so I should have tested the IP before proceeding to use it. Failing to check into this before hosting my email server using it was an oversight.
Thanks for the detailed feedback. For repeated blocks on authorized work, Support can review the exact safety message, model, timestamp and a redacted task description: https://help.openai.com/en/articles/20001326. Daybreak verification has separate requirements: https://help.openai.com/en/articles/20001259. We’ll pass along your feedback on false positives and verification; we can’t promise access or a timeline.