where reranking mattered: what are you seeing?

[modbench]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching.

I hit almost the same issue last night.

If you post your exact settings, people here can probably narrow it down pretty fast.