+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: vLLM vs SGLang for small teams on Threadripper box (/showthread.php?tid=714)
Pages:
1
2
vLLM vs SGLang for small teams on Threadripper box - cacheline - 10-03-2026
Maybe I'm missing something obvious.
I'm mostly working with Phi in LM Studio on 4090. The part I'm trying to understand is VRAM headroom.
Not looking for the perfect answer, mostly what worked in real use.
Re: vLLM vs SGLang for small teams on Threadripper box - cacheline - 10-03-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: vLLM vs SGLang for small teams on Threadripper box - batchcraft - 10-04-2026
Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?
Re: vLLM vs SGLang for small teams on Threadripper box - batchshed - 10-04-2026
Quote:Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: vLLM vs SGLang for small teams on Threadripper box - llmarch83 - 10-04-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: vLLM vs SGLang for small teams on Threadripper box - batchsize - 10-04-2026
Quote:What exact setup are you running? What model, what runtime, and roughly what context size?
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: vLLM vs SGLang for small teams on Threadripper box - deltapoint17 - 10-05-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. Turns out 'just one more test' was a lie.
Re: vLLM vs SGLang for small teams on Threadripper box - signalpeak - 10-08-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: vLLM vs SGLang for small teams on Threadripper box - cacheline - 10-08-2026
I hit almost the same issue last night.
Re: vLLM vs SGLang for small teams on Threadripper box - signalcraft14 - 10-09-2026
Interesting, I hadn't tried that. I ended up in about the same place using LM Studio. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.