10-19-2026, 12:34 AM
Quote:I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Once
Ran a quick check on my box. On A6000 with SGLang, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.