You're about to run DPO on 60,000 preference pairs, starting from your SFT model as both the policy and the reference.
A teammate asks what does and what value to start with.
You're about to run DPO on 60,000 preference pairs, starting from your SFT model as both the policy and the reference.
A teammate asks what does and what value to start with.