+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: best local open-weight model this week (/showthread.php?tid=448)
Pages:
1
2
best local open-weight model this week - audiorunner74 - 09-23-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate embeddings until they actually try to run this stuff for more than one afternoon.
Curious whether that lines up with what other people are seeing.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: best local open-weight model this week - labpath - 09-24-2026
worth checking the driver first.
Re: best local open-weight model this week - plainworks - 09-24-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: best local open-weight model this week - thetaplane - 09-24-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: best local open-weight model this week - quantsupply - 09-25-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: best local open-weight model this week - cobaltspark - 09-25-2026
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: best local open-weight model this week - sparrowghost - 09-25-2026
Quote:What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lo
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: best local open-weight model this week - localcraft17 - 09-27-2026
Good point. The pattern around embeddings has been pretty similar for me.
Re: best local open-weight model this week - vrambandit - 09-27-2026
Quote:Good point. The pattern around embeddings has been pretty similar for me.
worth checking the driver first.
Re: best local open-weight model this week - batchwire - 09-27-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.