+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: CLI helpers for local models with MLX (/showthread.php?tid=455)
CLI helpers for local models with MLX - batchshed - 09-25-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Would rather learn from your scars than collect my own all at once.
Re: CLI helpers for local models with MLX - rocmhouse90 - 09-25-2026
Can you fill in a bit more context? Are you testing this on 5090 or something else?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: CLI helpers for local models with MLX - mlxgarden - 09-28-2026
Appreciate the detail. I ended up in about the same place using llama.cpp.
Re: CLI helpers for local models with MLX - rocmpath - 09-28-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: CLI helpers for local models with MLX - cachebench - 09-28-2026
Thanks, that helps. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: CLI helpers for local models with MLX - mlxgarden - 10-01-2026
I think the bottleneck is somewhere else. I've seen people blame tokens/sec when the real issue was much more mundane.
Re: CLI helpers for local models with MLX - relayforge - 10-01-2026
That matches what I was seeing. I don't think your conclusion is far off.