+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: ROCm version that finally behaves for Gemma (/showthread.php?tid=1312)
Pages:
1
2
Re: ROCm version that finally behaves for Gemma - mangofield33 - 10-18-2026
same here on my side.
Re: ROCm version that finally behaves for Gemma - torchsmith25 - 10-18-2026
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.
Re: ROCm version that finally behaves for Gemma - marbletrack - 10-19-2026
Quote:I don't think that's the whole story. I get why that seems right, but on B200 the bottleneck felt elsewhere.
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.