10-08-2026, 07:28 PM
Did a bit of testing and thought I'd post the result here.
Short version: TensorRT-LLM behaved well for me, but only after I stopped ignoring tokens/sec.
Happy to post more details if anyone wants them.
Short version: TensorRT-LLM behaved well for me, but only after I stopped ignoring tokens/sec.
Happy to post more details if anyone wants them.
