+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits (/showthread.php?tid=378)
Apple unified memory limits - audiodesk - 09-20-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like quant drift decides everything downstream.
Wondering if this is just where the whole space is heading now.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: Apple unified memory limits - orbitline - 09-21-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: Apple unified memory limits - quietarch - 09-21-2026
Good point. I ended up in about the same place using TensorRT-LLM.