+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits on mini PC (/showthread.php?tid=30)
Apple unified memory limits on mini PC - torchshed - 08-25-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like prefix caching decides everything downstream.
Curious whether that lines up with what other people are seeing.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Apple unified memory limits on mini PC - thetaarch43 - 08-27-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: Apple unified memory limits on mini PC - mellowstream - 08-27-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: Apple unified memory limits on mini PC - plainember52 - 08-27-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: Apple unified memory limits on mini PC - labnest - 08-27-2026
Quote:Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was l
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Apple unified memory limits on mini PC - mellowstream - 08-27-2026
Quote:[quote]Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that move
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Apple unified memory limits on mini PC - plainember52 - 08-28-2026
My first guess would be this: On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.