+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits (/showthread.php?tid=791)
Apple unified memory limits - circuitwire58 - 10-05-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of Ollama and 4090 felt surprisingly usable once I tuned around continuous batching.
Feel free to tear it apart if you see something dumb.
Re: Apple unified memory limits - mlxgarden - 10-05-2026
Did a small comparison this afternoon. On Threadripper box with MLX, I saw basically the same behavior.
Re: Apple unified memory limits - visionyard23 - 10-06-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.