+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: TensorRT-LLM worth the setup time (/showthread.php?tid=62)
Pages:
1
2
Re: TensorRT-LLM worth the setup time - pixelcrate82 - 09-06-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: TensorRT-LLM worth the setup time - sysstation69 - 09-06-2026
Quote:Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it
Interesting, I hadn't tried that. The pattern around MoE serving has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
I'm happy to test one more variant if you want a second data point.