+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits with vLLM (/showthread.php?tid=825)
Pages:
1
2
Apple unified memory limits with vLLM - mlxgarden - 10-06-2026
Maybe I'm missing something obvious.
I'm mostly working with Phi in llama.cpp on GB200. The part I'm trying to understand is NVFP4.
Would appreciate practical advice rather than theory.
Re: Apple unified memory limits with vLLM - dockridge - 10-07-2026
Quote:Maybe I'm missing something obvious.
worth checking the driver first.
Re: Apple unified memory limits with vLLM - mlxgarden - 10-08-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher.
Re: Apple unified memory limits with vLLM - plainember52 - 10-08-2026
Maybe, but I had the opposite result. I've seen people blame consumer GPUs when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: Apple unified memory limits with vLLM - dockridge - 10-08-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal.
Re: Apple unified memory limits with vLLM - thetaarch43 - 10-08-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling.
Re: Apple unified memory limits with vLLM - torchdesk - 10-09-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: Apple unified memory limits with vLLM - batchsize - 10-09-2026
that fixed it for me too.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Apple unified memory limits with vLLM - cobaltlab - 10-10-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: Apple unified memory limits with vLLM - cudacrate - 10-10-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom. Once you know the plain path works, then add the faster path back in piece by piece.