+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: when TensorRT-LLM clearly wins with SGLang (/showthread.php?tid=232)
Pages:
1
2
when TensorRT-LLM clearly wins with SGLang - routecrate - 09-12-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once prefill/decode split becomes important.
What would you pick if you had to live with it for a few months?
Re: when TensorRT-LLM clearly wins with SGLang - modbench - 09-12-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: when TensorRT-LLM clearly wins with SGLang - syshouse - 09-12-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one mo
A couple things I'd check first. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. I keep promising myself this is the last rebuild.
Re: when TensorRT-LLM clearly wins with SGLang - tinycontext - 09-12-2026
Quote:[quote]That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding
I hit almost the same issue last night.
Re: when TensorRT-LLM clearly wins with SGLang - plainline - 09-13-2026
Quote:[quote][quote]That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of
same here on my side.
Re: when TensorRT-LLM clearly wins with SGLang - kernelmike - 09-13-2026
I hit almost the same issue last night.
Re: when TensorRT-LLM clearly wins with SGLang - sysstation69 - 09-13-2026
I tested this tonight. On GB200 with TensorRT-LLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: when TensorRT-LLM clearly wins with SGLang - rasterbench - 09-16-2026
I'd start with the simpler stack.
Re: when TensorRT-LLM clearly wins with SGLang - vectorhouse71 - 09-19-2026
Quote:I'd start with the simpler stack.
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: when TensorRT-LLM clearly wins with SGLang - batchcraft - 09-19-2026
following this because I'm in the same boat.