+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: best local runtime for mixed models (/showthread.php?tid=1360)
Pages:
1
2
Re: best local runtime for mixed models - vectorhouse71 - 10-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: best local runtime for mixed models - mellowwire98 - 10-19-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: best local runtime for mixed models - batchscope72 - 10-19-2026
I'd start with the simpler stack.
Re: best local runtime for mixed models - vectorhouse71 - 10-19-2026
Thanks, that helps. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: best local runtime for mixed models - dockridge - 10-19-2026
Good point. That matches what I saw on dual 3090.
Re: best local runtime for mixed models - edgeriver - 10-19-2026
Interesting, I hadn't tried that. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.