speculative decoding actually worth it with PyTorch

[mintforge]

My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.