Multimodal discussion on GB200

[fastlab]

following this because I'm in the same boat.

[tensorriver]

same here on my side.

[macrotinker]

A couple questions before I guess too hard. Are you testing this on A6000 or something else?

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[deepseeked]

That matches what I was seeing. The pattern around prefix caching has been pretty similar for me.

[mintstream9]

Maybe, but I had the opposite result. I've seen people blame HBM constraints when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.

[sparrowgrid]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?