A matrix lookup is an operation in which we retrieve a row or multiple rows from a matrix. Such lookup can be performed by simply indexing into the matrix. All modern Deep Learning frameworks support this operation but that has not always been the case.

Say we have a number of entities - we can represent each as a one hot encoded vector. When we pass such a representation to our model, it can look up relevant rows of data by performing matrix multiplication with one hot encoded vectors! By representing our entities not with indices, but as one hot encoded vectors, we transform matrix lookup into an operation that can be performed using a mathematical operation!

Understanding that these two ways of performing lookup are equivalent (with lookup by index being more computationally efficient) is essential - it will form the basis for figuring out what embeddings are and how they work.