You magnitude-pruned 50% of the weights in every linear layer, fine-tuned to recover accuracy, and stored the result as dense fp16 tensors full of zeros.

On an A100 the latency is unchanged.