+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: why did throughput drop after update for Kimi (/showthread.php?tid=990)
why did throughput drop after update for Kimi - labpath - 10-11-2026
I'm trying to sanity-check my setup.
I'm mostly working with Qwen in TensorRT-LLM on A6000. The part I'm trying to understand is speculative decoding.
Would appreciate practical advice rather than theory.
Re: why did throughput drop after update for Kimi - audioforge - 10-11-2026
Good point. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: why did throughput drop after update for Kimi - visionforge - 10-11-2026
Quote:Good point. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look c
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: why did throughput drop after update for Kimi - modrunner27 - 10-11-2026
What exact setup are you running? Are you testing this on dual 3090 or something else?
I'm happy to test one more variant if you want a second data point.
Re: why did throughput drop after update for Kimi - turboarch95 - 10-14-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: why did throughput drop after update for Kimi - nanogarden - 10-14-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: why did throughput drop after update for Kimi - oxideworks59 - 10-14-2026
Good point. That matches what I saw on Threadripper box. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: why did throughput drop after update for Kimi - thetashed24 - 10-14-2026
I hit almost the same issue last night.
Re: why did throughput drop after update for Kimi - thetaplane - 10-14-2026
same here on my side.
Re: why did throughput drop after update for Kimi - signalcraft14 - 10-15-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.