+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits (/showthread.php?tid=1157)
Apple unified memory limits - lablab - 10-14-2026
Did a bit of testing and thought I'd post the result here.
Short version: MLX behaved well for me, but only after I stopped ignoring router policy.
Nothing scientific here, just practical notes from a real setup.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: Apple unified memory limits - mellowstream - 10-14-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Apple unified memory limits - circuitwire58 - 10-14-2026
Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Apple unified memory limits - mlxgarden - 10-17-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: Apple unified memory limits - quietarch - 10-17-2026
Appreciate the detail. The pattern around workstation thermals has been pretty similar for me.
Re: Apple unified memory limits - cudacrate - 10-17-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at long context. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Apple unified memory limits - mlxgarden - 10-18-2026
I'd start with the simpler stack.
Re: Apple unified memory limits - cobaltworks25 - 10-18-2026
My first guess would be this: If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Apple unified memory limits - kernelrunner - 10-18-2026
Tried to reproduce it just now. On mini PC with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did.