+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: M3 Max for local use (/showthread.php?tid=686)
Pages:
1
2
M3 Max for local use - tinycontext - 10-02-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Gemma, Ollama, M3 Max, with a lot of attention on router policy.
Maybe useful as a starting point for newer folks.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Re: M3 Max for local use - tensorkite - 10-03-2026
Appreciate the detail. That matches what I saw on dual 3090.
Re: M3 Max for local use - vectorcrate98 - 10-06-2026
Good point. The pattern around token cost has been pretty similar for me.
Re: M3 Max for local use - quietarch - 10-06-2026
Quote:Good point. The pattern around token cost has been pretty similar for me.
I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: M3 Max for local use - circuitwire58 - 10-07-2026
That matches what I was seeing. That matches what I saw on B200. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: M3 Max for local use - torchshed - 10-08-2026
same here on my side.
Re: M3 Max for local use - labnest - 10-08-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
I'm happy to test one more variant if you want a second data point.
Re: M3 Max for local use - mlxgarden - 10-08-2026
My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: M3 Max for local use - sanescope - 10-09-2026
Appreciate the detail. The pattern around structured output has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: M3 Max for local use - visionyard23 - 10-10-2026
That matches what I was seeing. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.