Your eval pipeline samples 16 answers of about 300 tokens to each question, all behind the same 6,000-token few-shot prompt, and keeps the best one.
Profiling shows most GPU time and KV-cache memory go to prefill, since each of the 16 requests processes and stores the prompt separately.