what changed after the inference shift for Qwen

[kernelmike]

That matches what I was seeing. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looke

that fixed it for me too.

[edgeriver]

I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[tensorfield88]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

[kernelmike]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[stackarch]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed. I keep promising myself this is the last rebuild.

[macropilot]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?