im working on a project and i was wondering if there were any ‘best practices’ or established techniques to do this that i could measure my own against, forgive me if this is a dumb premise im self taught and its for my own personal entertainment; and until i get lucky enough to afford some formal teaching these projects are probably about as close as ill get. thanks for taking the time!
im pretraining a model with sentence embeddings represented in the data per sentence i was hoping to be able to perform an optimization technique on the total “tensor” of embeddings over a varying window of text related to its semantic structure and then teach the model to calculate true embedding positions based on just the optimized pseudo-tensor representation.
overall the idea is to create a model capable of extracting enough information out of a vector map to use the recent data Openai published to work on creating a ‘trained’ weight mqp based on wanted behaviors. i know its a lofty thing to shoot for and i dont really expect to be able to do that, but my primary motivation has been to contibute towards dealing with alignment as much as i can and to me this seems like a very clear and obvious trail to follow if the goal is to solve explainability and alignment.
