+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: throughput per watt reality check for Qwen (/showthread.php?tid=1250)
Pages:
1
2
Re: throughput per watt reality check for Qwen - kernelmike - 10-17-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: throughput per watt reality check for Qwen - stackarch - 10-17-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
Re: throughput per watt reality check for Qwen - lablab - 10-17-2026
I hit almost the same issue last night.
Re: throughput per watt reality check for Qwen - vectorhouse71 - 10-18-2026
Quote:Quote:I hit almost the same issue last night.
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: throughput per watt reality check for Qwen - kernelmike - 10-18-2026
Quote:Quote:Quote:I hit almost the same issue last night.
Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: throughput per watt reality check for Qwen - circuitworks90 - 10-19-2026
Quote:I hit almost the same issue last night.
That matches what I was seeing. That matches what I saw on B200. The biggest change for me was simplifying the stack instead of adding one more optimization layer.