+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: is RDNA worth the hassle yet with SGLang (/showthread.php?tid=659)
Pages:
1
2
is RDNA worth the hassle yet with SGLang - thetaember38 - 10-02-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on M3 Max, I ended up with decent performance, but the setup only really made sense once I adjusted around continuous batching.
Happy to post more details if anyone wants them.
Re: is RDNA worth the hassle yet with SGLang - quantshed - 10-03-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I keep promising myself this is the last rebuild.
Re: is RDNA worth the hassle yet with SGLang - cozykite - 10-04-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: is RDNA worth the hassle yet with SGLang - rocmmixer - 10-04-2026
I'd approach it in a boring order. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: is RDNA worth the hassle yet with SGLang - deltapoint17 - 10-04-2026
What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: is RDNA worth the hassle yet with SGLang - northhall88 - 10-06-2026
A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: is RDNA worth the hassle yet with SGLang - papermixer4 - 10-09-2026
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: is RDNA worth the hassle yet with SGLang - rocmmixer - 10-10-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: is RDNA worth the hassle yet with SGLang - tensorworks - 10-10-2026
That matches what I was seeing. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: is RDNA worth the hassle yet with SGLang - mintquill - 10-13-2026
Ran a quick check on my box. On M3 Max with vLLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.