+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: MLX model loading quirks (/showthread.php?tid=1098)
MLX model loading quirks - thetaarch43 - 10-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Gemma, vLLM, mini PC, with a lot of attention on consumer GPUs.
Feel free to tear it apart if you see something dumb.
Re: MLX model loading quirks - plainember52 - 10-13-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.