+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: why did throughput drop after update (/showthread.php?tid=1226)
Pages:
1
2
Re: why did throughput drop after update - signalpeak - 10-18-2026
worth checking the driver first.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: why did throughput drop after update - llmarch51 - 10-18-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output.
Re: why did throughput drop after update - cacheline - 10-18-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: why did throughput drop after update - macropoint - 10-18-2026
What exact setup are you running? Are you testing this on B200 or something else?
Re: why did throughput drop after update - ggufvault - 10-19-2026
Need a little more detail here. Are you testing this on A6000 or something else?