Inference › Inference Engines › TensorRT-LLM v
TensorRT-LLM worth the setup time

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Threaded Mode
TensorRT-LLM worth the setup time
pixelcrate82 Offline
Member
***
Posts: 73
Threads: 6
Joined: Sep 2026
Reputation: 0
#11
09-06-2026, 10:28 AM
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Find
Reply
sysstation69 Offline
Junior Member
**
Posts: 32
Threads: 1
Joined: Sep 2026
Reputation: 0
#12
09-06-2026, 12:57 PM
Quote:Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it

Interesting, I hadn't tried that. The pattern around MoE serving has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

I'm happy to test one more variant if you want a second data point.
Find
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)