+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: desktop runtimes for non-devs (/showthread.php?tid=1479)
Pages:
1
2
desktop runtimes for non-devs - benchplease - 10-18-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once MoE serving becomes important.
What would you pick if you had to live with it for a few months?
Re: desktop runtimes for non-devs - ggufwire22 - 10-18-2026
Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter
Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: desktop runtimes for non-devs - dockridge - 10-18-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: desktop runtimes for non-devs - vectorhouse71 - 10-18-2026
I'd approach it in a boring order. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: desktop runtimes for non-devs - cobaltspark - 10-18-2026
subscribed, curious where this lands.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: desktop runtimes for non-devs - mellowwire98 - 10-18-2026
Did a small comparison this afternoon. On GB200 with LM Studio, I saw basically the same behavior.
Re: desktop runtimes for non-devs - vectorcrate98 - 10-18-2026
Interesting, I hadn't tried that. I ended up in about the same place using LM Studio.
Re: desktop runtimes for non-devs - orbitbench6 - 10-18-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: desktop runtimes for non-devs - pixelspark - 10-19-2026
Quote:My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Not glamorous advice, but repe
Maybe, but I had the opposite result. I've seen people blame speculative decoding when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: desktop runtimes for non-devs - localcraft17 - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?