I’m learning PyTorch right now. Oftentimes my mind will create an analogy to make sense of mathematical concepts. This is a great tool unless my initial understanding of the concept is incorrect in which case, I’m screwed. I thought I’d test my understanding with GPT3 as my professor and me as the student asking questions and was very pleased with the results. I think what’s interesting about this in a more general sense is the idea of using a student’s current understanding of a concept as the starting point for teaching, rather than a starting point dictated by a teacher or syllabus.
I am your professor. Tell me what your understanding is and I will tell you if it’s correct or not. If it’s incorrect I will explain what is wrong with your understanding and then tell you the correct answer.
“”"
Student: When training a neural network using PyTorch, the gradient descent process can be understood using the following example. Let’s say a person is blindfolded and walking down a hill with an iPhone. At each step, Siri will tell them their elevation above sea level. They take one step to the left and Siri says “You’re at 80 feet above sea level”. Then they take one step to the right and Siri says “You’re 79 feet above sea level”. They take one step back and then decide that since a step to the right got them further down the hill, they will continue in that direction. Zeroing the gradients, in this example, is like when they step back to consider the best direction.
“”"
That is not quite correct. The zeroing of gradients is more like when you reset your iPhone’s elevation tracker back to zero. You do this so that you can accurately measure the change in elevation as you take each step.
“”"
Student: Describe something that the person would do in their journey that would be analogous to loss.backward().
“”"
There are a few things that the person could do that would be analogous to loss.backward(). One thing would be to keep track of how far they have walked in each direction. This would be like keeping track of the gradients. Another thing would be to keep track of how much elevation they have gained or lost in each direction. This would be like keeping track of the loss.
“”"
Student: Tell me the full story of the person who was blindfolded walking down the hill in the PyTorch training loop.
“”"
The person is blindfolded and walking down a hill. At each step, they take a measurement of their elevation above sea level. They then take a step in the direction that will get them further down the hill. They continue taking steps and measurements until they reach the bottom of the hill.