comparing models without fooling yourself with PyTorch

[cacheline]

Thanks, that helps. I ended up in about the same place using TensorRT-LLM.

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.