[hypothesis] finetuning for neural search and robot memory embedding

For background, Google (and others) have started experimenting with neural networks for search. At first, this simply using technologies like BERT to augment conventional search techniques. Then came the idea of replacing search altogether: you just embed all the possible search queries into your neural network and it becomes a QA repo for all the facts and information you want. Indeed, we see that GPT-3 already succeeds in this regard, especially with the latest INSTRUCT models. You can just plug in any question and it will spit out an answer. In this respect, it is a great form of memory, allowing for instant retrieval regardless of how much data there is to search.

In this case, it got the answer a little wrong. Claudia Black voices Morrigan, not Leliana.

What if you were to finetune GPT-3 with a list of linear, declarative, and episodic memories? This could allow a robot to possess instant retrieval of memories without conventional search mechanisms, such as a database or index search. In my book, Natural Language Cognitive Architecture, I proposed that a “shared database” would be the center of a robot’s brain. I have since speculated that such a database should be a blockchain so as to prevent tampering with the robot’s memories. I am now going to add another layer to this design, where the blockchain is used to record all memories in an immutable way, but that the memories should then be embedded into a finetuned model for instant recall.

I believe this two-step recall system is closer to how the human brain works.

Consider the fact that you can dredge up memories with very little cues. You might hear a voice or detect a scent and it can instantly dredge up vague, shadowy memories. But the more you think about it, you can sift through you memories and bring even higher detail back into your consciousness. By having an instant recall system (neural embedded memories) combined with a slower, but explicit system (blockchain), I think I can solve many of the problems I faced with NLCA - namely that the number of memories might grow forever, making it very difficult to search them all.

I was inspired by this recent Two Minute Papers video about NVIDIA’s neural representation of physical objects. It turns out that neural representations can be very fast, efficient, and dense. Perhaps we can embed more than just cloth physics and objects as neural representations - what if we can efficiently embed an entire lifetime’s worth of memories?

My next book Benevolent by Design will be coming out soon. I know what I’m working on next…

3 Likes