speculative decoding actually worth it with PyTorch

[mintforge]

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.