+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: throughput per watt reality check for DeepSeek (/showthread.php?tid=735)
throughput per watt reality check for DeepSeek - cacheline - 10-04-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: PyTorch on GB200, keeping batch size in mind from the start.
Maybe useful as a starting point for newer folks.
Re: throughput per watt reality check for DeepSeek - turboscope5 - 10-04-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
That matches what I was seeing. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: throughput per watt reality check for DeepSeek - edgeriver - 10-05-2026
Maybe, but I had the opposite result. I get why that seems right, but on B200 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: throughput per watt reality check for DeepSeek - northcraft90 - 10-06-2026
Appreciate the detail. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: throughput per watt reality check for DeepSeek - cachebench - 10-07-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. Not glamorous advice, but repeatable baselines save a lot of time.
Re: throughput per watt reality check for DeepSeek - cacheworks - 10-07-2026
worth checking the driver first.
Re: throughput per watt reality check for DeepSeek - marbletrack - 10-08-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: throughput per watt reality check for DeepSeek - kernelpeak - 10-08-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: throughput per watt reality check for DeepSeek - circuitworks90 - 10-08-2026
Quote:Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?