Bank
36 / 87
InitializationMultiple choiceEasy

You train a 30-layer ReLU network with no normalization layers, initialized with Glorot (Xavier) weights of variance , where is the layer's fan-in. The loss stays flat for many epochs.

Per-layer logging shows the activations' standard deviation shrinking by a constant factor at every layer, and the gradients reaching the first layers are tiny.

What would you change so the network trains?