Basic logic errors when programming

I’ve been using ChatGPT as a tool while programming and although it has been amazing in many respects I do struggle with it making “modifications” and “optimizations” to existing code that introduce bugs or suggests routines that clearly do not meet requirements.

Here is a simplified example that seems straight forward to me, but illustrates a core issue I have been having.


I have two potential states A and B that are independent and not exclusive. There are three tasks: 1, 2 and 3.

If A is true, do 1.
If B is true, do 2.
If neither A nor B is true, do 3.

Here is an example that implements this logic:

if A {
    do(1)
}
if B {
    do(2)
} 
If !A && !B {
    do(3)
}

Do see any simplification or optimizations that are functionally equivalent?


The consistent response I get back is like this:

Yes, there is a simpler way to express this logic:

if A {
do(1)
} else if B {
do(2)
} else {
do(3)
}

This implementation uses an if-else statement instead of three separate if statements. If A is true, it executes the first block and exits the if-else statement. If A is false, it checks if B is true, and if it is, it executes the second block and exits the if-else statement. If both A and B are false, it executes the third block. This implementation is functionally equivalent to the original implementation, but it is more concise and easier to read.


No, that is not functionally equivalent. It introduces an else in the conditions about A and B. If both A and B are true, my logic performs both tasks 1 and 2. What does your logic do in that case?

To which it responds:


You are correct, my previous suggestion would not be functionally equivalent to your original logic when both A and B are true.

To handle that case, we can use nested if statements as follows:

if A {
    do(1)
    if B {
        do(2)
    }
} else if B {
    do(2)
} else {
    do(3)
}

This implementation first checks if A is true, and if it is, it performs task 1 and checks if B is true. If both A and B are true, it performs task 2. If A is false, it checks if B is true and performs task 2 if it is. If both A and B are false, it performs task 3.

This implementation should be functionally equivalent to your original logic and handle all possible combinations of A and B correctly.


That is equivalent, but it is longer, confusing and harder to read than the original code.

A correct answer to the original question would have been:

if A {
    do(1)
}
if B {
    do(2)
} else if !A  {
    do(3)
}

That is shorter, has fewer conditions and is functionally equivalent.


You are absolutely correct! Your updated code is a better solution that is functionally equivalent to the original logic and also more concise and easier to read.

By using an if-else statement to check if B is true, and then checking if A is false in the else condition, we can cover all possible scenarios with just two conditions. Your solution is indeed shorter and more straightforward than the original logic.

Thank you for bringing this to my attention and providing a better solution.


This is a contrived example, but I think it does boil down some of the more complex logic issues I have been having where requirements or existing code that exhibits specific behavior is changed. I know it requires certain logical comprehension, but in cases like this the system makes invalid decisions.

Beyond pressing the thumbs down and trying to point out the flaws in responses as they come up, is there anything else I can do to help improve things?

2 Likes

ChatGPT is a text-prediction engine based on a large language model.

Because of that ChatGPT is simply generating text, blah, blah, blah without any expert or domain knowledge of the text it is generating.

It’s important to understand the many limitations of generative AI and how they generate text.

HTH

:slight_smile:

2 Likes

I will spend some more time with the codex models.
OpenAI Codex Models

Posing the same question to code-davinci-002 produced this output which is also incorrect.


A:

You can use the ternary operator:
do(A ? 1 : B ? 2 : 3)


Again, this is not functionally equivalent. Both models seem to really want to create an either or condition for A and B.

If A and B are both true then this statement calls do(1).

GPT-4 has a similar interaction, but doesn’t seem inclined to produce the more complex algorithm when making it functionally equivalent.

I have two potential states A and B that are independent and not exclusive. There are three tasks: 1, 2 and 3.

If A is true, do 1.
If B is true, do 2.
If neither A nor B is true, do 3.

Here is an example that implements this logic:

if A {
do(1)
}
if B {
do(2)
}
if !A && !B {
do(3)
}

Do see any simplification or optimizations that are functionally equivalent?


Yes, you can simplify the given code using if, else if, and else statements. Here is a more efficient way to implement the logic:

if A { 
     do(1) 
} else if B { 
     do(2) 
} else { 
     do(3) 
}

This implementation has the same functionality as the original code but is more efficient because it only needs to evaluate the conditions once. In the original code, it could potentially evaluate both A and B even when A is true. In the new implementation, it stops evaluating the conditions as soon as it finds the first true condition.


No, that is not functionally equivalent. It introduces an else in the conditions about A and B. If both A and B are true, my logic performs both tasks 1 and 2. What does your logic do in that case?


Apologies for the confusion. You are correct. The suggested code does not have the same functionality as the original code. In the case where both A and B are true, my suggested code will only perform task 1, whereas your original code will perform tasks 1 and 2.

Given your requirements, your original code is already efficient and does not need any further optimization:

if A {
    do(1)
}
if B {
    do(2)
} 
if !A && !B {
    do(3)
}

This code correctly performs task 1 if A is true, task 2 if B is true, and task 3 if neither A nor B is true. It also allows for both tasks 1 and 2 to be performed when both A and B are true.


Still not quite right. Here is a simplified implementation:

if A {
do(1)
}
if B {
do(2)
} else if !A {
do(3)
}

That is shorter, has fewer comparisons and is still functionally equivalent to the original code for all possible cases of A and B.


You are correct, and I apologize for not providing a better solution earlier. Your new implementation is indeed shorter, more efficient, and functionally equivalent to the original code:

if A {
    do(1)
}
if B {
    do(2)
} else if !A {
    do(3)
}

This code will perform task 1 if A is true, task 2 if B is true, and task 3 if neither A nor B is true. It also allows for both tasks 1 and 2 to be performed when both A and B are true, and it requires fewer condition evaluations compared to the original code.


I hear what ruby_coder says above about it being a text-prediction engine, but I keep having it revise code I’ve already got working with specific logic where it makes invalid assumptions about relationships between conditions.

In this contrived example, I provide a description that states A and B are not related or exclusive, give sample code that has specific behavior, and all the models so far have optimized it by adding an else or inferring a relationship between A and B that clearly changes the behavior of the logic.

Of course we can work for hours to find a way to get a language model to appear like it is a computing model on a case-by-case basis … but …

We can also take a screwdriver and use it do beat down nails.

Its simply best to use the right software tools for the job.

ChatGPT is based on a language model. It is an auto-completing text generation engine and has zero knowledge of “logic” because it is an LLM generating text, blah, blah, token-by-token and not an expert system AI.

HTH

:slight_smile: