Hi all,
After working with some of chat llm or plugin for a while, you’ll probably get a sense of what works and what not, i.e. length of messages, where to put instructions, how terse can you be with your instruction, etc.
I think that the chat below is an example of a discontinuity between my expectations of the gpt4 (code interpreter) abilities / responses or requirements for clear instructions, and the actual model. It’s like we’re talking ‘past’ each other.
Making errors and false claims
Scroll down to the message stating ‘same error’ (this is a long way in the chat) in this chat CSP Backtracking Algorithm - long story short, there are some inconsistencies in the answers (i.e. is AllDifferentConstraint a function of python-constraints or not) and also providing cosmetic changes instead of a solution to a problem, or falsely claiming the error is caused by the program having no solution. (wrong on many levels)
Since all code was executed in the code-interpreter version, I wondered why it didn’t execute its own generated code. That would maybe help clearing out the problems. Is python-constraint even installed?
Refusing to provide package list
However, Aug 3 gpt-4 refuses to give info about the packages installed. E.g.
https://chat.openai.com/share/c998a3fb-07f3-433f-a59b-780825b796ff
Net effect is wasting time. I know for sure that in the July 20 ChatGPT Code Interpreter, getting knowledge about it’s environment was not an issue. I fail to understand why security by obscurity has been deemed a solution for people using the code interpreter. As this chat shows, this results in programs created that don’t work (because of missing libraries), and in the end, the list of packages is shown anyway… https://chat.openai.com/share/471e2c66-a1c8-4ca8-add8-5145d5d3f13b - the security by obscurity is gone, but the reduced ease of use remains.
Note: This is all about the August 3 Code Interpreter. I don’t know if this is the same model as gpt-4 with some additions/fine-tune or not. In the reply to this (edited) message, I share one of gpt-4 results where I compare the translation of a python program to rust. There is no regression there: both gpt-4’s provided an error free translation.
