10-11-2026, 03:15 AM
Quote:Appreciate the detail. I ended up in about the same place using llama.cpp. I got better results once I stopped chasing one metric and just looked at w
Ran a quick check on my box. On M3 Max with TensorRT-LLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.