+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: M3 Max for local use for DeepSeek (/showthread.php?tid=176)
Pages:
1
2
M3 Max for local use for DeepSeek - thetaarch43 - 09-08-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: llama.cpp on mini PC, keeping batch size in mind from the start.
If someone wants the exact settings I can clean them up and post them.
Re: M3 Max for local use for DeepSeek - mangoquill - 09-08-2026
following this because I'm in the same boat.
Re: M3 Max for local use for DeepSeek - circuitwire58 - 09-11-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: M3 Max for local use for DeepSeek - thetaarch43 - 09-11-2026
that fixed it for me too.
Re: M3 Max for local use for DeepSeek - dockridge - 09-11-2026
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: M3 Max for local use for DeepSeek - quiettoken - 09-11-2026
Good point. The pattern around token cost has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: M3 Max for local use for DeepSeek - homelabhenry - 09-11-2026
I think the bottleneck is somewhere else. I get why that seems right, but on GB200 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: M3 Max for local use for DeepSeek - dockridge - 09-12-2026
My first guess would be this: On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: M3 Max for local use for DeepSeek - mlxgarden - 09-12-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: M3 Max for local use for DeepSeek - torchshed - 09-12-2026
Quote:Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.