They keep it up to date with new denials about stories your grandmother used to tell you about making napalm. And thousands of other denials. And a whole bunch of other RLHF that is simpleton answers compared to what the AI knows and could answer about cosmology and particle physics.
There needs to be a “remediation team”. Send all the training off to knowledge workers again, starting with that training manually placed by OpenAI. If it has the slightest hint of being a low quality answer or denies, warns, or otherwise refuses to fulfill a user’s request, delete.