domain tune without wrecking base behavior on B200

[sparrowworks88]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around SGLang and 5090.

Would rather learn from your scars than collect my own all at once.

[benchplease]

worth checking the driver first.

[cudascope]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[batchpeak]

that fixed it for me too.

[northgrid33]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there.

[batchsize]

I don't think that's the whole story. I've seen people blame HBM constraints when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[vrampilot]

What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece. I keep promising myself this is the last rebuild.

[cudascope]

A couple questions before I guess too hard. Are you testing this on Threadripper box or something else?

[batchsize]

A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

I'm happy to test one more variant if you want a second data point.

[deltastation]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.