Bank
5 / 87
EvaluationMultiple choiceMediumFree

You compare two chatbots by asking a strong model which of their two answers to each prompt is better, and rank them by pairs won.

Swapping the answer order changes the verdict on about a third of the pairs, and the judge leans toward whichever answer it read first.

Keeping the pairwise setup and the same judge, how would you make the comparison fair?