Bank
82 / 87
Fine-tuningMultiple choiceMedium

You fine-tune a 65B base model into an assistant with supervised examples. Doubling the training set from the same source doesn't improve the responses.

Most of the model's knowledge comes from pretraining; fine-tuning mainly teaches format and style.

Where would you put your next round of data effort?