+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: CLI helpers for local models with MLX (/showthread.php?tid=930)
Pages:
1
2
Re: CLI helpers for local models with MLX - batchshed - 10-13-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom.
Re: CLI helpers for local models with MLX - tinycontext - 10-13-2026
Interesting, I hadn't tried that. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: CLI helpers for local models with MLX - kernelgrid - 10-14-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.