Sam posed an interesting challenge in this video here:
I think it’s reasonable to say we’re not there yet. But what might be interesting is trying to do a more narrow version of it. Create a simulated universe which you can manipulate via text, and hook GPT4 up to it.
There’s actually a zillion things you can do with that, but I think the first fun thing to try to do would be to give the simulation some simplistic but interesting physical / chemical / biological properties and then give an autonomous agent using GPT4 a task to ‘invent’ some type of solution to a problem.
You’d probably want to start out as simple as possible and then slowly increase the complexity of the simulation to see at what point GPT4 starts to fail.