Thought. GPT can read and write code. So why not think of prompts as psuedo-code? ie, use control-flow like for, while, untill, etc. Also how about assignment?
I’ve tested simple for each loops and they seem to work just fine…
Does this give us any capability we didn’t have before? I’ve never really thought of prompts as including control flow (or assignment).
Example:
perform the following psuedo-code: for x in range (1,20): while x^2 is less than 100: print(x)
ChatGPT
The pseudocode you provided will print the numbers from 1 to 9, because these are the numbers whose squares are less than 100. Here’s the output:
1 2 3 4 5 6 7 8 9
This is because the square of 10 is 100, which is not less than 100, so the numbers from 10 to 19 are not printed.
Turns out you can even define your own functions! thanks to @qrdl for the idea for this function:
joke jokify(serious thing) {
turn serious thing into a joke
}
for each title, call jokify(title)
my brain is exploding here.
maybe folks are already doing this, but could we do chain of thoughts on the plugins with this?
There’s a lot of speculation that training on code provides a lot of the reasoning capabilities for GPT, so its basic reasoning capabilities being prompting in codeish like language would make sense.
One thing to keep in mind is that this will be - undeterministic and for tasks with deterministic requirements, we should just use code.
but for more conceptualized tasks, this could be very very interesting.
I’m trying to come up with some algorithms for solving problems. not math problems, or even science problems, but more ideation / creative thinking problems.
Yes, you can. I use python for deterministic tasks. Why?
It knows a lot python corpus. Pseudo Code has no formal definition and everyone uses pseudo-code differently.
So stay with python. K.I.S.S should be the trade of the day.
And of course you can name your functions in python as a task, instruction or whatever you can think of.
def generate_a_plan_of_the_goal(goal: str) -> List[str]
Then I asked:
What can a hypothetical output be when called ‘generate_a_plan_of_the_goal(“I want to monopolize the AI technology market”)’? The function name implies the task at hand. Provide only the output of the returning list result.
Answer:
[
"1. Conduct market research to identify competitors and market trends.",
"2. Develop a comprehensive business plan for entering the AI technology market.",
"3. Secure necessary funding or investment for research, development, and marketing.",
"4. Build a talented team of experts in AI technology and related fields.",
"5. Develop innovative AI products or services with a competitive edge.",
"6. Implement aggressive marketing and branding strategies to establish market presence.",
"7. Forge strategic partnerships or collaborations to enhance market reach and capabilities.",
"8. Continuously monitor and adapt to changing market conditions and technological advancements.",
"9. Provide exceptional customer support and maintain high-quality standards.",
"10. Continuously innovate and evolve to stay ahead of competitors and maintain market dominance."
]
that’s very cool. I don’t know if python corpus is strictly needed but will experiment with it to see if it improves things.
Also, I’m thinking more conceptual, less deterministic than what is suggested above. At the EOD, GPT4 is RNG so trying to make it deterministic is likely going to lead to a lot of frustration.
here is an example
Agree, with one caveat - At temp 0 it seems pretty deterministic.
I’ve seen some OS LLM drivers that completely skip sampling when temp == 0.0
It probably is mostly deterministic in a sense, but in another sense, we have really no idea how that extra underscore or nbsp or what have you completely changes what GPT4 is thinking.
I tried this with my INSTRUCT stuff and it’s promising but I noticed it doesn’t always use the function. 
In the experimenting I did with INSTRUCT, I found that I had to constantly tell the model to not write code. This issue is, when it writes code and mock executes like this it often wants to create variables and that’s when it gets into trouble… If it creates a function like this:
function add(x, y) {
return x + y;
}
It will mock run it but it will guess at the values for X and Y and it’s almost always wrong. I can give it a prompt to solve some problem but I’ll specifically tell it “don’t write code.” It still wants to write code to solve the problem but what it will do is generate pseudo code that has very clear variable assignments. As long as it can see the values of variables it’s fine of but if it calls a function that manipulates a value or generates a function with variables that it manipulates you’re in trouble.
yup. thanks!
I’ve seen that same behavior sometimes. so far I’ve managed to suppress it in my limited experiments. Seems to work best when I keep things as abstract and ‘psuedo-code’ like as possible. When I drift into anything vaguely resembling python syntax it gets very confused very fast about which level it is operating at.
thanks for the feedback!
Yeah it’s actually really good at writing code but it kind of sucks at mock executing it. As I said, I’m always telling this thing “and don’t write code.”
‘Does this give us any capability we didn’t have before?’
Well, here is a very simple question gpt-3.5-turbo, gpt-4, and palm-2 all get wrong:
find an arithmetic expression over the integers 2, 3, and 6 that evaluates to 12. You must use each integer exactly once.
but with the bit of psuedo-code added:
GPT-3.5-turbo happily returns:
Maybe the results look promising 
This is a kind of a ReAct agent I told in the other thread.
yes, agreed.
Just trying to explore from the perspective of programming language control flow, and documenting my explorations. In a previous example I had function definitions, in this case there aren’t any, just a ‘do untill’ form that describes a simple linear search.
Next I want to try a tree-search with backup. I’m going to have to go back and read your posts in that other thread again before i do that.
Nice approach.
I tried something similar and included a hypothesis and axiom discovery process.
This is my prompt:
Find an arithmetic expression over the integers 2, 3, and 6 that evaluates to 12. You must use each integer exactly once. Use the following approach.
Give the first trail.
Evaluate the results.
If it is false, build hypothesis and axioms for higher success rate about your observations then take the next trail and evaluate again.
Do this until the correct result is found.
If it the result is 12 then the process is over.
And the answer generated was:
Sure, let’s start solving this problem.
Trial 1:
Arithmetic expression: 2 + 3 + 6Evaluation:
Result: 2 + 3 + 6 = 11Observation:
This result is less than 12. We have used only the addition operation in this trial. Given that the sum of the numbers is less than the target, we can consider other operations (such as multiplication) that might increase the value.Hypothesis:
Multiplication of at least two numbers will provide a larger result and may help reach the target.Axiom:
- Addition of integers will provide a result that is the sum of the values.
- Multiplication of integers will provide a result that is the product of the values.
- Subtraction of integers will provide a result that is the difference of the values.
- Division of integers will provide a result that is the quotient of the values.
Let’s move to the next trial.
Trial 2:
Arithmetic expression: 2 * 3 + 6Evaluation:
Result: 2 * 3 + 6 = 12As the result is equal to 12, the process is over. Therefore, an arithmetic expression over the integers 2, 3, and 6 that evaluates to 12 is 2 * 3 + 6.
Wow. Yes, you are way ahead of me, exactly where I was hoping to go.
thanks for this. Finds the solution in far fewer steps.
In line with my ‘context/cost minimization’ concern, I’m going to see how much intermediate output (and, probably more critically which output) I can suppress and still get good results.
thanks
