10-04-2026, 06:47 PM
I tested this tonight. On Threadripper box with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
I'm happy to test one more variant if you want a second data point.
I'm happy to test one more variant if you want a second data point.
