Question 4/5 fast.ai v4 lecture 6

Answer

Say we have a number of entities - we can represent each as a one hot encoded vector. When we pass such a representation to our model, it can look up relevant rows of data by performing matrix multiplication with one hot encoded vectors! By representing our entities not with indices, but as one hot encoded vectors, we transform matrix lookup into an operation that can be performed using a mathematical operation!

Understanding that these two ways of performing lookup are equivalent (with lookup by index being more computationally efficient) is essential - it will form the basis for figuring out what embeddings are and how they work.

Relevant part of lecture