CUDA update broke my throughput

[phaseport36]

Appreciate the detail. I ended up in about the same place using llama.cpp. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[kernelmike]

What exact setup are you running? Are you testing this on 5090 or something else?

[dockember]

that fixed it for me too.