Bank
67 / 87
DataMultiple choiceMedium

You pretrain your own base models, and you have 40B tokens of instruction and synthetic QA data. Fine-tuning the finished base on all of it for two epochs lifts instruction-following scores, but knowledge benchmarks fall several points below the base model.

How would you use the data instead?