09-24-2026, 12:35 AM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between Ollama and TensorRT-LLM for Qwen. The interesting difference for me has been token cost.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
The part that surprised me was how much the boring details mattered more than the model choice.
I've gone back and forth between Ollama and TensorRT-LLM for Qwen. The interesting difference for me has been token cost.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
The part that surprised me was how much the boring details mattered more than the model choice.