09-24-2026, 08:56 AM
Quote:Ran a quick check on my box. On A6000 with MLX, I saw basically the same behavior. The main thing that moved was latency, not quality.
That matches what I was seeing. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.