10-01-2026, 05:27 PM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between TensorRT-LLM and TensorRT-LLM for Llama. The interesting difference for me has been tokens/sec.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.
I've gone back and forth between TensorRT-LLM and TensorRT-LLM for Llama. The interesting difference for me has been tokens/sec.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.