An established technique for teaching optimized vector paths?

im working on a project and i was wondering if there were any ‘best practices’ or established techniques to do this that i could measure my own against, forgive me if this is a dumb premise im self taught and its for my own personal entertainment; and until i get lucky enough to afford some formal teaching these projects are probably about as close as ill get. thanks for taking the time!

im pretraining a model with sentence embeddings represented in the data per sentence i was hoping to be able to perform an optimization technique on the total “tensor” of embeddings over a varying window of text related to its semantic structure and then teach the model to calculate true embedding positions based on just the optimized pseudo-tensor representation.

overall the idea is to create a model capable of extracting enough information out of a vector map to use the recent data Openai published to work on creating a ‘trained’ weight mqp based on wanted behaviors. i know its a lofty thing to shoot for and i dont really expect to be able to do that, but my primary motivation has been to contibute towards dealing with alignment as much as i can and to me this seems like a very clear and obvious trail to follow if the goal is to solve explainability and alignment.

Hi, and welcome to the community.

This is a really good question. I don’t have an answer, but in a very limited experiment using an FAQ as an example, I assumed the embeddings that I needed to generate had to have as much related content as possible to provide a broader vector. I was wrong - the embeddings needed to be discrete and focused. The more each embedding was about a key idea, the better the similarity matching worked. It was my duh moment. :wink: