ROCm version that finally behaves

[thetaember38]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of vLLM and Threadripper box felt surprisingly usable once I tuned around NVFP4.

Feel free to tear it apart if you see something dumb.

[quantshed]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.

[dockworks23]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[batchyard]

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[plainline]

I'd start with the simpler stack.

[thetaember38]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.

[mintquill]

My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.