10-05-2026, 10:08 PM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between PyTorch and TensorRT-LLM for Mistral. The interesting difference for me has been speculative decoding.
Trying to avoid rebuilding the whole thing twice.
I've gone back and forth between PyTorch and TensorRT-LLM for Mistral. The interesting difference for me has been speculative decoding.
Trying to avoid rebuilding the whole thing twice.