aiquizzes
Question 9/13 fast.ai v3 lecture 11

What is LAMB?

Answer

It is ADAM but avaraged out over each layer.

Relevant part of lecture

supplementary material

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes