Bank
48 / 87
InitializationShort answerMedium

You're about to train a 40-layer ReLU MLP with no normalization layers, and a failed run costs a day of compute.

How would you choose the initialization, and how would you check it before the run?