Persistent state for programming

[jimbo2112]

I’ve started trying to get 3.5 to do some coding and then switched to 4.0.

My observations are that there is no persistence in the development conversation. You can ask 3.5 to always do something and it will fail to do it directly after asking it to do it. An example is when I asked for code to be included to allow ctrl-x to quit at anytime in case it gets into a infinite loop. It would forget to include in code iterations over and over.

It’s like you are speaking to a fully functioning programmer with complete amnesia.

I got better results with 4.0 although that was not perfect

So my question is 4-fold.

1. Do you have the same experience?
2. Are there ways around this?
3. Are there better programming environments or custom GPTs for coding?
4. Is there any plan, or custom GPT’s for an environment that will persist with data in a session and then outside of the session so as to allow an assistant who will learn cumulatively from the user?

Pretty sure this needs to be the way ahead, but also imagine it would be a huge resource requirement.

Tell me your thoughts!

[Macha]

What exactly do you mean by “persistent state” here? Do you mean like, a persistent terminal that’s running in real-time?

Code Interpreter is single-shot by design. meaning, you ask it one query, it will load the container (I think it’s some kind of special Jupyter notebook iirc?), run the code, then close the container. It can not maintain a persistent terminal for a wide variety of reasons.

Are you calling code interpreter via API in a thread, or are you using ChatGPT? That will help me in providing you more detailed help.

Also, some more direct examples of what’s going on would be helpful here, like some screenshots.

If you want persistent programming help, Github Copilot is the best thing I could suggest, but please be aware, I wouldn’t call it “persistent”. Just more integrated in the script you’re editing.

[TonyAIChamp]

Assistant APIs kinda do that, but they are still in beta and not very reliable and flexible.

Most of the developers track state themselves now, I believe.

[jimbo2112]

What exactly do you mean by “persistent state” here? Do you mean like, a persistent terminal that’s running in real-time?
Excuse my inexperience - I mean by holding information in memory for at least the session you are in. It constantly forgets explicit instructions over and over

Code Interpreter is single-shot by design. meaning, you ask it one query, it will load the container (I think it’s some kind of special Jupyter notebook iirc?), run the code, then close the container. It can not maintain a persistent terminal for a wide variety of reasons.
OK, got it, that makes sense, but it needs to get a better solution soon as the experience is quite poor. Worse than that, It’s inconsistent. It remembers some stuff but not others

Are you calling code interpreter via API in a thread, or are you using ChatGPT? That will help me in providing you more detailed help.
Chat GPT

Also, some more direct examples of what’s going on would be helpful here, like some screenshots.
I think you’ve properly explained the lack of persistence, but appreciate the offer to support

If you want persistent programming help, Github Copilot is the best thing I could suggest, but please be aware, I wouldn’t call it “persistent”. Just more integrated in the script you’re editing.
**Cheers, I’ll take a look

I think the key thing is to have an application that retains cumulative information about the user so the interactions become more useful the more you use it**

[jimbo2112]

Assistant APIs kinda do that, but they are still in beta and not very reliable and flexible.

Most of the developers track state themselves now, I believe.

Have you got any example APIs I could play with on this? I really think this is the way forward for a lot of use cases and will add a good volume of quality training data for next generation AI

[Macha]

Don’t apologize, we are all here to learn and help each other, no matter the experience level.

So, when I hear the word “session” in this context, I think of a conversation thread. It’s context window (i.e. the closest definition to its memory) is bound within the conversation and any persistence between conversations are done using either custom instructions or vector databases.

If by session you mean “the thing it does when it runs code in the conversation”, then I think you might still be misunderstanding what it actually is and what it’s doing.

The code runner is not a terminal. ChatGPT cannot read your keystrokes, so asking for a ctrl+x keystroke should not be possible nor attempted in the context of ChatGPT. That would be a pretty significant security risk for OpenAI. Containerizing the code execution in a non-persistent way has benefits far outweighing any inconveniences, and most code doesn’t need to be run in a persistent way anyway. If it is, just have it write the code for you to whip up in a terminal, cat the output to a .txt file, give it the resultant output, and it should modify the code it produced fine.

I’m using ChatGPT daily for frequent CLI and development work, and it’s quite powerful and helpful. Does it give me 100% accurate commands each and every time 24/7? Not quite, but once you understand the “flow” of how to use ChatGPT, it’s extremely effective at helping you.

Especially with coding/development work, it’s best to ask questions with it iteratively, meaning start small, and have it add to the code with each conversation.
It’s easier to show this than explain it:

When it runs analysis mode, check the blue box to see its work, shown here:

It should pop out a screen like this:

You can continuously refine, adjust, and scaffold your code this way while it tests out the code.

If you’re trying to make it run infinite loops: don’t.