Question 3/7 fast.ai v4 lecture 8
For NLP, is it better to unfreeze many layers at once or gradually unfreeze layers, one by one?
Answer
Gradual unfreezing works better.
Relevant part of lecture
supplementary material
Unsupervised Data Augmentation for Consistency Training - a great collection of data augmentation techniques for text