I have found this kind of architecture can be experimented with if one thinks of prompts as programs. For example, even gpt-4 can’t solve
find an arithmetic expression over the integers 4 and 6 that evaluates to 24. You can only use each integer once
but, if I add a loop that seems very similar to your idea, but at the prompt level, gpt-3.5 has no problem. Steps 3/4 seem similar to your control-gate, I think?
find an arithmetic expression over the integers 2, 3, and 6 that evaluates to 12. You must use each integer exactly once. use the following problem-solving method:
- generate a trial expression
- evaluate the trial expression.
- test if the result of step 2 equals 12
- if test is positive, respond with trial expression and STOP.
- Otherwise, add trial expression to the list of expressions tried
Loop until test is positive