Mistral AIDataMultiple choiceMediumCopy prompt

When and how would you use the high-quality data?

1. AFine-tune on it after pretraining ends, at a fresh high learning rate.
2. BUpsample it in the final phase while the learning rate decays to zero.
3. CRestart pretraining with it mixed evenly into the corpus from step one.
4. DHold it out entirely and only use it to pick the best checkpoint.