+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: QLoRA resource reality check with TensorRT-LLM (/showthread.php?tid=1409)
QLoRA resource reality check with TensorRT-LLM - sparrowbrook - 10-17-2026
Did a bit of testing and thought I'd post the result here.
Short version: SGLang behaved well for me, but only after I stopped ignoring prefix caching.
Nothing scientific here, just practical notes from a real setup.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: QLoRA resource reality check with TensorRT-LLM - northgrid33 - 10-17-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: QLoRA resource reality check with TensorRT-LLM - benchplease - 10-18-2026
Good point. The pattern around quant drift has been pretty similar for me.