NVIDIA driver regression

[edgecoast]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: TensorRT-LLM on 7900 XTX, keeping HBM constraints in mind from the start.

If someone wants the exact settings I can clean them up and post them.

[latentgarden]

That matches what I was seeing. The pattern around token cost has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[kernelmike]

That matches what I was seeing. The pattern around token cost has been pretty similar for me. The biggest change for me was simplifying the stack inst

I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.

[mintforge]

[quote]That matches what I was seeing. The pattern around token cost has been pretty similar for me. The biggest change for me was simplifying the sta

That matches what I was seeing. I don't think your conclusion is far off.

[thetapath]

Can you fill in a bit more context? Are you testing this on B200 or something else?

[dockember]

Can you fill in a bit more context? Are you testing this on B200 or something else?

Need a little more detail here. Are you testing this on 4090 or something else?

[sparrowkite53]

Can you fill in a bit more context? Are you testing this on B200 or something else?

What worked for me was pretty simple. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[edgecoast]

Can you fill in a bit more context? Are you testing this on B200 or something else?

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.

[emberrunner]

Can you fill in a bit more context? Are you testing this on B200 or something else?

That matches what I was seeing. That matches what I saw on Threadripper box. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[latentgarden]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.