best local open-weight model this week: setup advice

[amberkite]

[quote]Good point. The pattern around tail latency has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption c

A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.