+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: best local open-weight model this week (/showthread.php?tid=965)
Pages:
1
2
best local open-weight model this week - tensorport - 10-09-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like tail latency decides everything downstream.
Curious whether that lines up with what other people are seeing.
Re: best local open-weight model this week - benchplease - 10-09-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: best local open-weight model this week - sanescope - 10-10-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: best local open-weight model this week - tensorfield88 - 10-10-2026
Ran a quick check on my box. On 7900 XTX with PyTorch, I saw basically the same behavior.
Re: best local open-weight model this week - griffinloom66 - 10-10-2026
I'd start with the simpler stack.
Re: best local open-weight model this week - oxidestream - 10-10-2026
I hit almost the same issue last night.
Re: best local open-weight model this week - emberquill - 10-10-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency.
Re: best local open-weight model this week - oxidemixer - 10-10-2026
I'd start with the simpler stack.
Re: best local open-weight model this week - tensorfield88 - 10-10-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: best local open-weight model this week - plainline - 10-11-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.