Blackwell deployment chatter on Threadripper box

[kernelmike]

worth checking the driver first.

[phaseport36]

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

If you post your exact settings, people here can probably narrow it down pretty fast.

[amberpatch25]

What exact setup are you running? Is this local-only or are you routing anything externally?

[audioriver]

I hit almost the same issue last night.

[latentgarden]

I tested this tonight. On 7900 XTX with llama.cpp, I saw basically the same behavior. I expected the model choice to matter more than it did.

[orbitbench6]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.