+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: best local stack on mac right now with Ollama (/showthread.php?tid=1201)
Pages:
1
2
best local stack on mac right now with Ollama - mlxgarden - 10-14-2026
I'm trying to sanity-check my setup.
My current setup is PyTorch with dual 3090, and I keep running into HBM constraints.
If you've tested something similar, what did you keep and what did you stop bothering with? At this point the real bottleneck is my patience.
Re: best local stack on mac right now with Ollama - mlxgarden - 10-14-2026
I don't think that's the whole story. I've seen people blame INT4 when the real issue was much more mundane.
Re: best local stack on mac right now with Ollama - visionyard23 - 10-15-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
I'm happy to test one more variant if you want a second data point.
Re: best local stack on mac right now with Ollama - labnest - 10-15-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: best local stack on mac right now with Ollama - sanescope - 10-15-2026
Quote:What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: best local stack on mac right now with Ollama - turboyard94 - 10-16-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: best local stack on mac right now with Ollama - torchstation - 10-17-2026
Quote:Can you fill in a bit more context? Is this local-only or are you routing anything externally?
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local stack on mac right now with Ollama - cudacrate - 10-18-2026
I hit almost the same issue last night.
Re: best local stack on mac right now with Ollama - sanescope - 10-18-2026
Quote:I hit almost the same issue last night.
I'd approach it in a boring order. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local stack on mac right now with Ollama - quiettoken - 10-18-2026
Quote:Quote:I hit almost the same issue last night.
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.