best local open-weight model this week

[audiorunner74]

This keeps coming up in conversation so I figured I'd start a thread.

One thing I keep noticing is that people underestimate embeddings until they actually try to run this stuff for more than one afternoon.

Curious whether that lines up with what other people are seeing.

If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.

[labpath]

worth checking the driver first.

[plainworks]

What exact setup are you running? Is this local-only or are you routing anything externally?

[thetaplane]

What exact setup are you running? Is this local-only or are you routing anything externally?

I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[quantsupply]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.

[cobaltspark]

What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[sparrowghost]

What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lo

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.

If you post your exact settings, people here can probably narrow it down pretty fast.

[localcraft17]

Good point. The pattern around embeddings has been pretty similar for me.

[vrambandit]

Good point. The pattern around embeddings has been pretty similar for me.

worth checking the driver first.

[batchwire]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.