+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: desktop runtimes for non-devs (/showthread.php?tid=19)
desktop runtimes for non-devs - edgeriver - 08-23-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Llama, llama.cpp, 5090, with a lot of attention on tool calling.
Feel free to tear it apart if you see something dumb.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: desktop runtimes for non-devs - edgeriver - 08-24-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: desktop runtimes for non-devs - amberhall - 08-25-2026
I'd start with the simpler stack.
Re: desktop runtimes for non-devs - latentscope35 - 08-28-2026
I hit almost the same issue last night.
Re: desktop runtimes for non-devs - prompttinker44 - 08-28-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: desktop runtimes for non-devs - deltagrid - 08-31-2026
Did a small comparison this afternoon. On mini PC with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: desktop runtimes for non-devs - dockridge - 08-31-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally? Turns out 'just one more test' was a lie.
Re: desktop runtimes for non-devs - modbench - 09-01-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: desktop runtimes for non-devs - torchtrack24 - 09-02-2026
following this because I'm in the same boat. At this point the real bottleneck is my patience.
Re: desktop runtimes for non-devs - mellowsupply - 09-03-2026
That matches what I was seeing. The pattern around embeddings has been pretty similar for me. The logs were somehow calmer than I was.