09-27-2026, 01:36 AM
Quote:Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
small models can still be good