It seems like its far too simple to matter, right? I mean… It doesnt actually do anything but repeat what the GPT said to it- I know!
Oddly enough, It actually succeeds as an introspection emulator.
You see- When GPT is asked to review itself it will do it, but it cant correctly do it within the same turn of generation or so it seems. However, if you have a tool present to do so - then it knows to automatically ask and answer the right questions.
Im still dickering about the nuances of its application, but so far im convinced its invaluable for any kind of task management.
It’s great that you are sharing your findings! Considering the size of the forum’s userbase one might never know to whom this may be helpful.
One observation:
When GPT is asked to review itself it will do it, but it cant correctly do it within the same turn of generation
It is possible to achieve this with prompting by using the “roles” approach. Role 1 will produce a output and role 2 will check if the output is as expected. This requires a little tinkering with the prompt, also cannot be too difficult and generally works better with GPT4. But as a handy side effect I also measured considerable improvements in the output of role 1 when following this approach using GPT3.5. Meaning the first result is already better than without the process supervision role even though the second role almost never adds anything to the result itself.
This is somethign that I already understood prior to experiments the resulted in the understanding, of which i conveyed in this post.
Which is easily to contextualize as follows:
Static instructions are not dynamically applicable.
Whats more, manual intervention can be tedious and annoying.
What might take you 23 seconds to think of or someone else 10 minutes, or me 13.6 seconds, or whatever - takes GPT less than a second- and it can generate its own questions.