09-28-2026, 11:02 PM
Quote:I tested this tonight. On Threadripper box with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.
Good point. The pattern around tokens/sec has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
