+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: best local open-weight model this week (/showthread.php?tid=969)
Pages:
1
2
best local open-weight model this week - quantghost - 10-10-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: Ollama on mini PC, keeping prefix caching in mind from the start.
Maybe useful as a starting point for newer folks.
This is mostly for local work rather than public serving.
Re: best local open-weight model this week - sanevault - 10-10-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: best local open-weight model this week - rocmrunner40 - 10-10-2026
I'd start with the simpler stack.
Re: best local open-weight model this week - thetagrid15 - 10-10-2026
What worked for me was pretty simple. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local open-weight model this week - oxidemixer - 10-10-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: best local open-weight model this week - southtinker - 10-13-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: best local open-weight model this week - sanevault - 10-13-2026
My first guess would be this: If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: best local open-weight model this week - mangohouse - 10-13-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local open-weight model this week - cudaworks - 10-13-2026
A couple questions before I guess too hard. Are you testing this on B200 or something else?
Re: best local open-weight model this week - rocmrunner40 - 10-13-2026
What exact setup are you running? Are you testing this on dual 3090 or something else?
I'm happy to test one more variant if you want a second data point.