Your 70B model's decoding is memory-bound, so you add a small draft model: it proposes four tokens, one at a time, and the big model scores all four in one forward pass for about the cost of generating one token.

Left to right, each draft token is kept with probability $\min (1,p/q)$, where $p$ and $q$ are the big and draft models' probabilities for it. The first rejected token is resampled from the big model's leftover distribution and the rest of the draft is discarded; if all four survive, the same pass yields a fifth token. The output follows the big model's distribution exactly, and you measure about a 2 to 3x speedup.