+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: ROCm version that finally behaves for Gemma (/showthread.php?tid=1312)
Pages:
1
2
ROCm version that finally behaves for Gemma - benchplease - 10-16-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Phi, LM Studio, 5090, with a lot of attention on long context.
If someone wants the exact settings I can clean them up and post them.
Re: ROCm version that finally behaves for Gemma - audiopath66 - 10-17-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: ROCm version that finally behaves for Gemma - dockworks23 - 10-17-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: ROCm version that finally behaves for Gemma - torchsmith25 - 10-18-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: ROCm version that finally behaves for Gemma - southpatch - 10-18-2026
I hit almost the same issue last night.
Re: ROCm version that finally behaves for Gemma - rocmriley - 10-18-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: ROCm version that finally behaves for Gemma - torchsmith25 - 10-18-2026
Quote:Quote:I don't think that's the whole story. I get why that seems right, but on B200 the bottleneck felt elsewhere.
What worked for me was pretty simple. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: ROCm version that finally behaves for Gemma - northgrid33 - 10-18-2026
subscribed, curious where this lands.
Re: ROCm version that finally behaves for Gemma - papermixer4 - 10-18-2026
Good point. The pattern around VRAM headroom has been pretty similar for me.
Re: ROCm version that finally behaves for Gemma - quantshed - 10-18-2026
I don't think that's the whole story. I get why that seems right, but on B200 the bottleneck felt elsewhere.