+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: when TensorRT-LLM clearly wins (/showthread.php?tid=214)
Pages:
1
2
Re: when TensorRT-LLM clearly wins - routecrate - 09-16-2026
that fixed it for me too.
Re: when TensorRT-LLM clearly wins - sparrowcraft - 09-16-2026
Good point. I don't think your conclusion is far off.
Re: when TensorRT-LLM clearly wins - rocmriley - 09-16-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: when TensorRT-LLM clearly wins - sparrowhall - 09-16-2026
Good point. That matches what I saw on mini PC.
Re: when TensorRT-LLM clearly wins - kernelmike - 09-16-2026
I hit almost the same issue last night.
Re: when TensorRT-LLM clearly wins - sysstation69 - 09-16-2026
Quote:I hit almost the same issue last night.
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did. At this point the real bottleneck is my patience.