It’s only because at the time I only had 2 arms and 10 fingers
1 Like
Embeddings having a temperature would be a bit like being able to turn up the heat and the universe instantly expanding 
1 Like
If anything, you want all your embedding temps to be zero.
Just to maintain the same embedding value for the same given input.
A high embedding temp would imply a high randomness on the embedding vector components, much higher than the annoying randomness introduced by the asynchronous GPU clocking.
People would not be happy with high embedding temps.
2 Likes