where reranking mattered for Mistral

[quietarch]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.

[orbitgarden]

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[kernelghost]

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.