best small quant that still feels smart

[thetarunner]

[quote]A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. O

I tested this tonight. On M3 Max with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.