+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: best local stack on mac right now (/showthread.php?tid=1383)
Pages:
1
2
best local stack on mac right now - latentgarden - 10-17-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on 7900 XTX, I ended up with decent performance, but the setup only really made sense once I adjusted around HBM constraints.
Happy to post more details if anyone wants them.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: best local stack on mac right now - mellowstream - 10-17-2026
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: best local stack on mac right now - cacheline - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: best local stack on mac right now - thetaarch43 - 10-18-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Re: best local stack on mac right now - visionyard23 - 10-18-2026
Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Thanks, that helps. That matches what I saw on Threadripper box. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: best local stack on mac right now - mellowstream - 10-18-2026
Quote:Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Can you fill in a bit more context? Are you testing this on Threadripper box or something else?
I'm happy to test one more variant if you want a second data point.
Re: best local stack on mac right now - labnest - 10-18-2026
Quote:Quote:Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
I'm happy to test one more variant if you want a second data point.
Re: best local stack on mac right now - mellowstream - 10-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
I'm happy to test one more variant if you want a second data point.
Re: best local stack on mac right now - dockridge - 10-19-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
I tested this tonight. My result was mixed: first run looked bad, second run looked normal.
Re: best local stack on mac right now - kernelpeak - 10-19-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.