+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: TensorRT-LLM worth the setup time with SGLang (/showthread.php?tid=984)
TensorRT-LLM worth the setup time with SGLang - torchshed66 - 10-09-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about tokens/sec.
Simple and boring is a valid answer.
This is mostly for local work rather than public serving.
Re: TensorRT-LLM worth the setup time with SGLang - macrotinker - 10-11-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: TensorRT-LLM worth the setup time with SGLang - northcraft90 - 10-12-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: TensorRT-LLM worth the setup time with SGLang - northgrid33 - 10-12-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
Re: TensorRT-LLM worth the setup time with SGLang - routecrate - 10-12-2026
I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere.
Re: TensorRT-LLM worth the setup time with SGLang - macrotinker - 10-12-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: TensorRT-LLM worth the setup time with SGLang - kernelgrid - 10-13-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.