ScalingYou pretrain with a cosine learning-rate schedule sized for a fixed token budget. Partway through, you decide to train on more data, but the cosine's decay is already planned around the original count.
Restarting for every new budget costs too much, and the final loss must match a cosine run.