It looks like GPT-4-32k is rolling out

I’ve been doing this to solve some RW tasks and the results have been competitive with what another expert at my level could do.

GPT4 is really superior at pattern matching and then massaging the text it finds to fit your prompts.

On top of that, it can only do some very limited reasoning, but nothing particularly competitive with what a human can do.

The other problem is that it lacks the total context usually required to competently do a task.

The key to optimal GPT4 usage I am beginning to believe is leveraging the pattern matching and finding a way to inject your own reasoning skills. The best way to do this is via ‘reverse prompting’ which is getting GPT4 to ask you questions as if you were the expert.

This does two things - it injects your superior human reasoning skills and it helps complete the context that GPT4 is missing. The results are really great.

This still fails for some things, like complex documents (eg, a technical design document), but it works for others which are not as congitively complex (eg, a requirements document)

That said, even when it fails, it still asks a lot of great questions which help you realize what you forgot to consider.