+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: why did throughput drop after update (/showthread.php?tid=524)
why did throughput drop after update - thetaplane - 09-28-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between vLLM and vLLM for Kimi. The interesting difference for me has been reranking.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: why did throughput drop after update - promptpoint - 09-28-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: why did throughput drop after update - phasestream53 - 09-29-2026
Did a small comparison this afternoon. On A6000 with PyTorch, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.