why did throughput drop after update

[signalpeak]

worth checking the driver first.

If you post your exact settings, people here can probably narrow it down pretty fast.

[llmarch51]

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output.

[cacheline]

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[macropoint]

What exact setup are you running? Are you testing this on B200 or something else?

[ggufvault]

Need a little more detail here. Are you testing this on A6000 or something else?