comparing models without fooling yourself on GB200

[kernelarch]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.

[prompttinker44]

Appreciate the detail. That matches what I saw on GB200. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[batchfield55]

A couple questions before I guess too hard. Are you testing this on M3 Max or something else?

I'm happy to test one more variant if you want a second data point.

[northgrid33]

Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.