+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: best local open-weight model this week (/showthread.php?tid=1187)
Pages:
1
2
best local open-weight model this week - cudahouse - 10-13-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with consumer GPUs.
Curious whether that lines up with what other people are seeing.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: best local open-weight model this week - bytesupply76 - 10-15-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local open-weight model this week - nanopatch - 10-18-2026
My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: best local open-weight model this week - signaltrack44 - 10-18-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: best local open-weight model this week - dockridge - 10-18-2026
subscribed, curious where this lands.
Re: best local open-weight model this week - stackpilot - 10-18-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: best local open-weight model this week - latentpeak22 - 10-18-2026
Quote:My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.
I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: best local open-weight model this week - kernelarch - 10-18-2026
subscribed, curious where this lands.
Re: best local open-weight model this week - cozygrid12 - 10-19-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: best local open-weight model this week - tinycontext - 10-19-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?