What would you change first?

1. ARaise the learning rate, since updates this size mean the steps are too small.
2. BAdd weight decay, since the weights are too small relative to their updates.
3. CUse a larger batch, since the ratio shows the gradients are mostly noise.
4. DLower the learning rate until the ratio sits near $10^{-3}$, then retune.