+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: throughput per watt reality check (/showthread.php?tid=1040)
Pages:
1
2
Re: throughput per watt reality check - quantyard - 10-17-2026
Good point. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: throughput per watt reality check - orbitgarden - 10-17-2026
I'd start with the simpler stack.
Re: throughput per watt reality check - bytelab - 10-17-2026
that fixed it for me too.
Re: throughput per watt reality check - papermixer4 - 10-17-2026
I'd start with the simpler stack.
Re: throughput per watt reality check - routeforge64 - 10-17-2026
Good point. I don't think your conclusion is far off.
Re: throughput per watt reality check - kernelpeak - 10-17-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. The logs were somehow calmer than I was.