Tensor cores actually helping here on B200

[dockember]

My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[amberpatch25]

I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.

[kernelmike]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[orbitbench6]

I'd start with the simpler stack.