Does L2 regularization have a regularizing effect when used with BatchNorm?

Answer

No.

Relevant part of lecture

supplementary material

L2 Regularization versus Batch and Weight Normalization L2 Regularization and Batch Norm - a blog post by David Wu, summary of the blog post: