+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: Any point in NPUs yet (/showthread.php?tid=590)
Any point in NPUs yet - turboscope - 09-29-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around MLX and M3 Max.
Simple and boring is a valid answer.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: Any point in NPUs yet - latentcraft - 09-30-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Any point in NPUs yet - audioriver - 09-30-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Any point in NPUs yet - tensorriver - 09-30-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: Any point in NPUs yet - routearch - 09-30-2026
that fixed it for me too.
Re: Any point in NPUs yet - lablab - 09-30-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Any point in NPUs yet - kernelhouse70 - 10-03-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.