Temporal/Linear Coding with Embeddings?

[curt.kennedy]

Have you thought about just time stamping the embedding, as an additional metadata field?

You can mix this into retrieval by using something like RRF, where you mix time and correlation into your overall ranking.

Same sort of thing for positional data, but here you have the positional offset, which can be mixed in as well.

I had more details on “aging” the timestamps as well, to declutter things and boost performance.

Biggest difficulty in developing LLM apps

Good point. You should bin it according to the amount of information, per topic even, that you are ingesting. So I would implement this as a timestamp quantized to whatever level. And you can change the quantization as the data ages. For example, say you have a hot topic, or story, with minute by minute updates coming in. So quantize a UNIX timestamp to the nearest minute. I would use the floor operation (see below). But this just adds some zeros to the end, and quantizes the output. As …

So the application would be news stories, where newer is better, and you want good separation with RRF, and so you demote (age) the timestamp, on a schedule, by zero filling the UNIX timestamp from the right.

Also with the repetitive “alarm” scenario, of “feed your dog at 8am”. The aging process puts all the information in the same time bucket over time, and you can detect common/repetitive events, and treat them differently if you wish. But the detection/clustering rate is determined by your aging schedule.

So you quantize time over time, similar to how our brains work.