You deepen a plain convolutional network with batch normalization from 18 to 34 layers. Top-1 validation error goes from 27.94% to 28.54%, and the 34-layer net's training error is higher than the 18-layer net's throughout training.
Forward activations and backward gradient norms both look healthy.