+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: best local runtime for mixed models for Mixtral (/showthread.php?tid=75)
best local runtime for mixed models for Mixtral - llm_wren - 08-31-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about reranking.
I'm especially interested in mistakes worth avoiding.
This is mostly for local work rather than public serving.
Re: best local runtime for mixed models for Mixtral - batchscope72 - 08-31-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.