+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: CLI helpers for local models with MLX (/showthread.php?tid=930)
Pages:
1
2
CLI helpers for local models with MLX - batchshed - 10-08-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like VRAM headroom decides everything downstream.
Curious whether that lines up with what other people are seeing.
Re: CLI helpers for local models with MLX - cachebench - 10-10-2026
Appreciate the detail. The pattern around tool calling has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: CLI helpers for local models with MLX - audionode8 - 10-10-2026
A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: CLI helpers for local models with MLX - mlxgarden - 10-12-2026
worth checking the driver first.
Re: CLI helpers for local models with MLX - rocmriley - 10-12-2026
worth checking the driver first.
Re: CLI helpers for local models with MLX - vectorport - 10-13-2026
My first guess would be this: On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: CLI helpers for local models with MLX - kernelgrid - 10-13-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: CLI helpers for local models with MLX - rocmhouse90 - 10-13-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: CLI helpers for local models with MLX - audionode8 - 10-13-2026
following this because I'm in the same boat.
Re: CLI helpers for local models with MLX - deltagrid - 10-13-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.