Question 3/7 fast.ai v4 lecture 8

For NLP, is it better to unfreeze many layers at once or gradually unfreeze layers, one by one?

Answer

Gradual unfreezing works better.

Relevant part of lecture

supplementary material

Unsupervised Data Augmentation for Consistency Training - a great collection of data augmentation techniques for text