10-19-2026, 05:33 AM
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4.
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
routing beats arguing over one perfect model