You move a 1.3B model (24 layers, $d_{\mathrm{model}}=2048$) from 2K to 32K-token sequences for a long-context phase, with the same GPUs and kernels.

Tokens per second fall to about a third. You compute MFU as $6N$ FLOPs per token, with $N$ the 1.3B parameter count, times tokens per second over the GPUs' peak, so MFU looks like it collapsed.