+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: throughput per watt reality check: worth it? (/showthread.php?tid=518)
throughput per watt reality check: worth it? - cacheplane98 - 09-26-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Phi using MLX on A6000. The headline result was better than I expected, but NVFP4 still ended up being the limiting factor.
Nothing scientific here, just practical notes from a real setup.
Re: throughput per watt reality check: worth it? - routecraft - 09-26-2026
Interesting, I hadn't tried that. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.