09-23-2026, 01:09 AM
Did a bit of testing and thought I'd post the result here.
Short version: TensorRT-LLM behaved well for me, but only after I stopped ignoring quant drift.
Happy to post more details if anyone wants them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Short version: TensorRT-LLM behaved well for me, but only after I stopped ignoring quant drift.
Happy to post more details if anyone wants them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
curious and caffeinated