+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Performance Tuning (https://www.datasculpting.ai/inference/forumdisplay.php?fid=42)
+--- Thread: what changed after the inference shift for Qwen (/showthread.php?tid=1140)
Pages:
1
2
Re: what changed after the inference shift for Qwen - kernelmike - 10-18-2026
Quote:That matches what I was seeing. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looke
that fixed it for me too.
Re: what changed after the inference shift for Qwen - edgeriver - 10-18-2026
I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: what changed after the inference shift for Qwen - tensorfield88 - 10-18-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
Re: what changed after the inference shift for Qwen - kernelmike - 10-18-2026
Quote:I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: what changed after the inference shift for Qwen - stackarch - 10-19-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed. I keep promising myself this is the last rebuild.
Re: what changed after the inference shift for Qwen - macropilot - 10-19-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?