serving on Blackwell

[kernelmike]

What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[rasterbench]

Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane.

[tensorfield88]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching.