best small quant that still feels smart

[nanobrook]

A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you

What worked for me was pretty simple. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time. Turns out 'just one more test' was a lie.

[thetarunner]

[quote]A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. O

I tested this tonight. On M3 Max with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.

[mangofield33]

following this because I'm in the same boat.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.