+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: new here, what's your stack with SGLang (/showthread.php?tid=77)
Pages:
1
2
Re: new here, what's your stack with SGLang - tensorkite - 09-09-2026
Interesting, I hadn't tried that. The pattern around INT4 has been pretty similar for me.
Re: new here, what's your stack with SGLang - rocmrunner40 - 09-09-2026
Quote:Interesting, I hadn't tried that. The pattern around INT4 has been pretty similar for me.
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift. Once you know the plain path works, then add the faster path back in piece by piece.
Re: new here, what's your stack with SGLang - vramghost - 09-12-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: new here, what's your stack with SGLang - rocmplane - 09-12-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: new here, what's your stack with SGLang - cacheplane98 - 09-13-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.