throughput per watt reality check

[quantyard]

Good point. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[orbitgarden]

I'd start with the simpler stack.

[bytelab]

that fixed it for me too.

[papermixer4]

I'd start with the simpler stack.

[routeforge64]

Good point. I don't think your conclusion is far off.

[kernelpeak]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. The logs were somehow calmer than I was.