I think what @anon37218972 is referring to is under the umbrella of Deep NeuroEvolution.
But as it turns out, OpenAI already employs their own evolution strategies (OpenAI ES). Just look them up.
These date back to 2017, so not sure how much they are used now, but here is a blog post.