+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: why did throughput drop after update (/showthread.php?tid=1226)
Pages:
1
2
why did throughput drop after update - batchsize - 10-14-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around vLLM and 4090.
Would rather learn from your scars than collect my own all at once.
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: why did throughput drop after update - signalcraft14 - 10-15-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: why did throughput drop after update - batchsize - 10-17-2026
My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: why did throughput drop after update - cacheline - 10-17-2026
Quote:My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorou
subscribed, curious where this lands.
Re: why did throughput drop after update - mangofield33 - 10-17-2026
I don't think that's the whole story. I get why that seems right, but on B200 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: why did throughput drop after update - nanogarden - 10-18-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal.
Re: why did throughput drop after update - audioforge - 10-18-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: why did throughput drop after update - batchsize - 10-18-2026
that fixed it for me too.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: why did throughput drop after update - ggufvault - 10-18-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
Re: why did throughput drop after update - batchsize - 10-18-2026
Need a little more detail here. What model, what runtime, and roughly what context size?