Each SFT example is a 1,800-token news article followed by a 120-token summary, and you average the cross-entropy over every token in the sequence.
Training loss drops quickly, but summary quality on held-out articles barely moves.
Each SFT example is a 1,800-token news article followed by a 120-token summary, and you average the cross-entropy over every token in the sequence.
Training loss drops quickly, but summary quality on held-out articles barely moves.