09-25-2026, 03:41 PM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between LM Studio and TensorRT-LLM for Mistral. The interesting difference for me has been prefix caching.
Trying to avoid rebuilding the whole thing twice.
I've gone back and forth between LM Studio and TensorRT-LLM for Mistral. The interesting difference for me has been prefix caching.
Trying to avoid rebuilding the whole thing twice.
measure twice, benchmark three times