So what I’ve been doing with Chat GPT-3 is pasting in summaries of classes I’ve taught, and then querying it. The issue is that once I’ve pasted in about 5 pages of summaries (roughly 2,400 words) it starts failing to retrieve information that was pasted in earlier in the thread. So basically, it can only recall the information from the latest 2,400 words or so of the thread.
Does anyone know if the pro version of Chat GPT-3 will be more powerful? This memory limitation makes it a no-go for a lot of use cases (e.g. semantic search through archives of meeting minutes, checking a novel draft for plot holes).
I’ve noticed similar limitations. Have you tried giving a name to each of the uploads? “Part 1” “Part 2” etc – then ask it to incorporate part 1 and part 2… I’ve had some success that way.
There are definitely limitations. Not sure it will work but have you tried naming each time period?
In other words, start by just dropping in the the 2019 dates and explaining to ChatGP… “these are events that relate to 2019. Let’s name them “2019 events” Do you understand that these events relate to the year 2019? Do you understand that I will share events that relate to 2020 and 2021 next?”
Once it understands, then add the 2020 events – name it and make sure it understands, again… then, repeat for 2021 - naming and making sure it understands.
Once you do that, try asking the question… it just might work.
That wouldn’t work, because it completely forgets the info from earlier in the thread. For example, the 2019 data described a trip to watch Japanese kabuki. But even when I ask Chat GPT-3 whether there is any information about kabuki, it says there is nothing about kabuki. I tried querying it a bunch of ways, and got the same result.
OpenAI models such as davinci have a limit of 4000 tokens, which would get pretty close to the limitation of ~2000 words you saw.
ChatGPT probably summarizes things internally to improve this “limit”, but as soon as you have a lot of condensed information, it will get lost quickly.
Absolutely awful. Today it remembered yesterday’s script, wrote it again with modifications, then an app mistakenly reported fonts missing, ChatGPT suggested installing fonts in the system, I said there were plenty, ChatGPT recommended a command prompt to query my system for fonts installed, I pasted the list, ChatGPT said something generic about the list and regardless of prompts couldn’t put it back into context
I totally agree about increasing the tokens. Pretty worthless otherwise. You can’t plot a movie if it forgets the title of the movie. ChatGPT is not even able to render the name under which the topic is being saved
I’m experiencing a similar problem. I share AlexTully’s viewpoint and was already considering the same solution. It would be beneficial if a premium version with varying limit tokens was introduced. Personally, I would be interested in purchasing an unlimited version.
We are also now offering dedicated instances for users who want deeper control over the specific model version and system performance. By default, requests are run on compute infrastructure shared with other users, who pay per request. Our API runs on Azure, and with dedicated instances, developers will pay by time period for an allocation of compute infrastructure that’s reserved for serving their requests.
Developers get full control over the instance’s load (higher load improves throughput but makes each request slower), the option to enable features such as longer context limits, and the ability to pin the model snapshot.
Dedicated instances can make economic sense for developers running beyond ~450M tokens per day. Additionally, it enables directly optimizing a developer’s workload against hardware performance, which can dramatically reduce costs relative to shared infrastructure. For dedicated instance inquiries, contact us.
The memory capacity is not exactly something they can just increase arbitrarily. It’s a limitation of the way they trained the model. The older, smaller models had even less capacity (about half). To increase the capacity significantly would require upgrading the model and retraining it from scratch, pretty much. To my understanding anyway.
When I first started using it, I used it to write an entire 200 Page e- book I spent days and days chatting with it and developing the book and it was amazing and it worked great (I should mention using chat GPT to write a book you’re still doing about 90% of the work). Then the news broke and it became the hot thing and the developers must have changed something about how handles resources and how much resources it retains per conversation because it’s slowly deteriorated over the point to now a few months later it’s become almost useless because it can only hold context of your previous input during a conversation. You can’t even compose an email because you’ll ask two or three questions and it forgets everything from the first two question and only responds to the last question. Then you have to start building this huge context to paste in each time and then you start approaching the letter limit. Subscribing to premium doesn’t change this. Very disappointing tool.
ChatGpt has become forgetting what you said to it in the same conversation, for example I send the first part and the second part, then it forgets the first part after sending second part. In the last month or before this problem does not exist. I and many are wondering whether it is being improved or made weaker?
