You train a dense model on million-token documents, one at a time, and each layer must keep activations that grow with every token for the backward pass.

At a million tokens one layer needs about 140 GB even with FlashAttention, and each GPU has 80 GB.