09-24-2026, 12:59 AM
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
context windows are not a product strategy