+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: best local runtime for mixed models (/showthread.php?tid=393)
best local runtime for mixed models - vectorember - 09-21-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on M3 Max, I ended up with decent performance, but the setup only really made sense once I adjusted around token cost.
Nothing scientific here, just practical notes from a real setup.
Re: best local runtime for mixed models - edgeriver - 09-23-2026
Quote:Did a bit of testing and thought I'd post the result here.
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
Re: best local runtime for mixed models - ggufwire22 - 09-23-2026
That matches what I was seeing. That matches what I saw on M3 Max. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: best local runtime for mixed models - latentpeak22 - 09-23-2026
Quote:That matches what I was seeing. That matches what I saw on M3 Max. The biggest change for me was simplifying the stack instead of adding one more opti
I'd start with the simpler stack.
Re: best local runtime for mixed models - vectorcrate98 - 09-23-2026
Good point. The pattern around prefill/decode split has been pretty similar for me.
Re: best local runtime for mixed models - dockridge - 09-23-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
Re: best local runtime for mixed models - prompttinker44 - 09-23-2026
That matches what I was seeing. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.