+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: ROCm version that finally behaves (/showthread.php?tid=719)
ROCm version that finally behaves - thetaember38 - 10-04-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of vLLM and Threadripper box felt surprisingly usable once I tuned around NVFP4.
Feel free to tear it apart if you see something dumb.
Re: ROCm version that finally behaves - quantshed - 10-05-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: ROCm version that finally behaves - dockworks23 - 10-08-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: ROCm version that finally behaves - batchyard - 10-08-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: ROCm version that finally behaves - plainline - 10-09-2026
I'd start with the simpler stack.
Re: ROCm version that finally behaves - thetaember38 - 10-10-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.
Re: ROCm version that finally behaves - mintquill - 10-13-2026
My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.