Ohh noo not an expert… Everyone here is super competent!!! (PS still havent gotten to ur book yet sorry!!)
Oh yes yes YES! Fantastic thoughts! Ye text search is kinda “solved” I guess. You can use advanced data structures like tries, trays etc which solves the text search problem efficiently. 16. Strings - YouTube
Obviously some use a combination of tree methods and hash tables / arrays which speeds up search even more.
Another obvious way is use Count Vectorization. Ie u preprocess text and split it into tokens, and store the sparse array of boolean values YES/NO. The issue is this fails for say searching if a sentence exists in a corpus. THis only works for 1 word. You can try N-grams, but its inefficient.
For memories - this is the holy grail!! The issue as u mentioned is we need to somehow “retrieve” old memories, “summarize/compress” it, and “update” the memory state.
I did mention random ideas in A theoretical novel writing tool - #20 by danielhanchen which I “tried” to show some ideas on keeping a running memory for Transformer based models.