+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: best local runtime for mixed models with Ollama (/showthread.php?tid=749)
best local runtime for mixed models with Ollama - fusedhall - 10-04-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once token cost becomes important.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: best local runtime for mixed models with Ollama - rocmcrate26 - 10-04-2026
I'd start with the simpler stack.
Re: best local runtime for mixed models with Ollama - mellowwire98 - 10-04-2026
What exact setup are you running? Are you testing this on 4090 or something else?
Re: best local runtime for mixed models with Ollama - orbitbench6 - 10-04-2026
Quote:What exact setup are you running? Are you testing this on 4090 or something else?
subscribed, curious where this lands.
Re: best local runtime for mixed models with Ollama - vectorember - 10-05-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.
Re: best local runtime for mixed models with Ollama - amberhall - 10-05-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local runtime for mixed models with Ollama - mellowwire98 - 10-06-2026
What worked for me was pretty simple. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: best local runtime for mixed models with Ollama - amberhall - 10-07-2026
That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: best local runtime for mixed models with Ollama - vectorember - 10-08-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.
Re: best local runtime for mixed models with Ollama - fusedhall - 10-08-2026
What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.