comparing models without fooling yourself with PyTorch

[benchplease]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. A lo

subscribed, curious where this lands.