10-10-2026, 04:42 PM
Quote:Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter mor
Ran a quick check on my box. On A6000 with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.
context windows are not a product strategy