Hi everyone,
I’m not a developer or ML researcher - just a long-time user who’s spent a lot of time interacting with GPT-4 in extended, multi-layered conversations.
After enough sessions, I started noticing a limitation:
the model handles context windows well, but it doesn’t retain semantic continuity -
the ability to hold on to what matters, conceptually, beyond just the current thread.
Here’s the idea I arrived at (non-engineer perspective):
LLMs store recent tokens and patterns well,
but they don’t retain persistent conceptual relevance across sessions.
What if we added a Semantic Memory Layer -
a lightweight structure where the model stores high-salience meaning nodes
extracted from attention-weighted clusters in the conversation?
These nodes wouldn’t be raw tokens or logs,
but abstracted representations of significance -
key ideas that carried weight through patterns like recurrence, user emphasis, focus density, or shifts in attention dynamics.
Over time, the model could build a conceptual graph of what matters to a user -
allowing it to return not just to prior wording,
but to underlying meaning.
I imagine this sitting somewhere between attention output and final decoding -
not a full symbolic system, but enough to allow for symbolic continuity and
user-specific meaning accumulation.
I don’t have the technical background to build or benchmark this -
just sharing an idea that emerged through real, prolonged interaction.
Thanks for reading,
Irina