benchmarks that map to real use

[vramport]

Did a bit of testing and thought I'd post the result here.

After a few runs on A6000, I ended up with decent performance, but the setup only really made sense once I adjusted around VRAM headroom.

Nothing scientific here, just practical notes from a real setup.

[tensorport]

Did a bit of testing and thought I'd post the result here.

A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. I keep promising myself this is the last rebuild.

[deltastation]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.

[signalpeak]

What exact setup are you running? Are you testing this on A6000 or something else?

[edgequill]

Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[audiopoint18]

subscribed, curious where this lands.

[cozywire]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. I keep promising myself this is the last rebuild.

[southspark]

worth checking the driver first.

[kernelgrid]

subscribed, curious where this lands.

[routeline23]

Appreciate the detail. The pattern around HBM constraints has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

If you post your exact settings, people here can probably narrow it down pretty fast.