+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: why did throughput drop after update (/showthread.php?tid=1132)
why did throughput drop after update - nanogarden - 10-12-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of llama.cpp and 4090 felt surprisingly usable once I tuned around structured output.
Feel free to tear it apart if you see something dumb.
Re: why did throughput drop after update - thetaplane - 10-13-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once you know the plain path works, then add the faster path back in piece by piece.
Re: why did throughput drop after update - mangofield33 - 10-14-2026
Good point. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: why did throughput drop after update - routearch - 10-14-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: why did throughput drop after update - ggufvault - 10-14-2026
What exact setup are you running? Are you testing this on A6000 or something else?
Re: why did throughput drop after update - batchsize - 10-14-2026
Quote:What exact setup are you running? Are you testing this on A6000 or something else?
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: why did throughput drop after update - thetashed24 - 10-15-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.