Hi I’m trying to use ChatGPT 3.5-turbo model to identify faults, status and other values into a register. For example, I provide the IA with a context about the register definition (hipotetical register from my invention). The context contains the register definition, what is used for, the address, the lenght and all the bit fields definition. I indicate that is bigendian format and everything. But when I ask the IA to analyze a value and try to identify the meaning bit by bit it not just fails when mapping the bits with my register definition but it also fails in a basic tasks like saying that a bit is clear or set when it is not.
prompt: The value in hexadecimal of the 16bit length “register” is 0x0200. Identify bit by bit what faults are caused and the meaning of each one set.
Response: Sure, in the value 0x0200 which is 0b0000010000000000 in binary the bit 0, bit 1 and bit 8 are set which means: bla bla bla (if those bits were set, the answer is correct).
So if I continue asking for different values sometimes is rigth and sometimes is wrong (most of times is wrong). I tried to give it some examples, tell it what is the bit 0, how it is incremented, that more than one bit could be set and always fails. What concerns to me is that even if the values conversion are good the IA is not able to identify if a bit is set or clear even when the conversion and the bits idenfitication is done right. That makes the analysis to crash.
The only way it does not fails is giving an example of all the possible values in the register (I tried this in a 4 bits register), but imagine the mess for a 16bit or 32bit register.
Does anybody has detected this issue, is this a bug or this is something beyond the capabilities of the ChatGPT (which is weird cause is just simply binary aritmetic)? Maybe I’m wrong with the way I’m giving it context, the format presented (just text and descriptions) or the way I’m doing the prompts. Hope anybody can help me, this is hanging me in the investigation I’m doing for my thesis