I am a school teacher and I’m trying to set up a session, with either o1 or o1 pro where I can give it multiple assignments particular to each student and also other documentation that can help me make better decisions with interventions for my students. I’m a special education teacher, so my class sizes are usually between 12 to 20 students. The class I started with is 13 students. I gave it specific instructions to remember the information particular to each student and create a class roster as I gave it information. What I found out is that after I had given it all of the information for all 13 students, o1 and o1 pro could only remember 7 of the students. If I re-entered in information for one of the students it had forgotten, it would remember that student, but then forget another, always keeping the class roster to 7 students.
If I told it that class roster at first, it would remember all 13 students, but it would forget stuff about the students once I got beyond entering information for more than 7 students. Is there a limit on o1 as to much much it can remember? If not, is this just a temporary bug?
Your issue might be with the context window and how OpenAI manages this. They do manage it a bit smarter than this (as I understand it) but each request is up to total 200k tokens (About 800kb of data) with 100k tokens (400kb of data returned)
So as you are adding more data it is running out of room.
(I have found conflicting information on context window sizes but I believe the number above is the current correct number)
Conflicting Information
What’s the context window for OpenAI o1 models?
In ChatGPT, the context windows for o1 and o1-mini is 32k.
I think what you’re saying is that when I type or paste something I want it to process, the query or request needs to be 200,000 tokens or less. Is that accurate?
If so, the issue I’m having is that something that it remembers at one point, it forgets as I add more information to the session. So… I can ask it to tell me everything about the first 5 students in the session once I’ve added the 5 student’s information. However, once I add the rest, having added information for 13 students, one document at a time, it will forget anything we had talked about in regards to the first 5 students that we were talking about before. It won’t remember the conversation we had about them. Is that normal?
This is not an exact number (or process). I can’t tell you exactly how ChatGPT manages the data but the model itself accepts 200k tokens yes and only sends 100k back.
Indeed also bare in mind that it may also ‘hallucinate’ on some of the data. So may not always return all the data exactly as it should do.
It is entirely normal for chats not to grasp the full context (but still give highly accurate and extremely useful results).
Thank you. I do have a different chat set up for each student. That has been really helpful for managing each student individually. The thing that I want to do is be able to have one chat that knows everything about all of the students, and then can tell me how to create groups of students that are at a similar level of understanding, and then create lesson plans, activities, interventions for that group of students.
Thank you for the recommendation on the course. I’ll start it right away!
The “forgetfulness” when using o1 is enhanced by the large token context consumption by o1 models. The information provided in a past chat turn is distracted from and may even expire from the large amount of “reasoning” done about an input. It is a model for problem-solving, but gpt-4 in ChatGPT may be better for this conversational memory needing attention.
There is an additional feature, “actions”, in creating a GPT (a custom personality, available in ChatGPT plus), where you can employ an external API and its database you program as a tool the AI can call on when useful.
The database itself sounds like your solution for keeping records - and you can build a web interface instead of chatting with a bot that can make errors. There are companies where outsourced education management, up to grading and plagiarism checks, is their whole business.
I would also rethink if submitting personal information of minors without consent to an AI corporation, that retains data for model training, which is shared with 3rd party global contractors, isn’t an overstep…
I usually explain that I redact all personal information from any documents that I upload. It’s a lot of work to do, but it is worth it. Additionally, I have opted out of sharing my conversations to make it better. Everything is confidential and above board, but I don’t have time to explain that every single time I write something. Why do you just assume the worst? Why not ask something instead of assuming?
