+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: CLI helpers for local models (/showthread.php?tid=229)
CLI helpers for local models - modgarden - 09-12-2026
I'm trying to sanity-check my setup.
My current setup is LM Studio with A6000, and I keep running into VRAM headroom.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
Re: CLI helpers for local models - amberpatch25 - 09-13-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: CLI helpers for local models - rocmriley - 09-13-2026
I tested this tonight. On 4090 with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did.