adapter stacking

[torchsmith25]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around TensorRT-LLM and A6000.

Simple and boring is a valid answer.

[northgrid33]

subscribed, curious where this lands.

[batchpeak]

worth checking the driver first.

[deltastation]

same here on my side.

[localtinker]

same here on my side.

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.

[benchplease]

A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[quietsupply11]

That matches what I was seeing. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[deltastation]

same here on my side.

[tensorport]

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

[cacheline]

That matches what I was seeing. That matches what I saw on 4090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.