throughput per watt reality check for Mistral

[bytelab]

That matches what I was seeing. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumpt

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.